Collaborative Research


Published on

Slides from my Collaborative Research workshop from Webstock 2014.

Buy the book: Just Enough Research

Published in: Design, Technology, Education

Collaborative Research

  1. Collaborative Research with Erika Hall
  2. Hello!
  3. I have a question...
  4. Do you enjoy being right?
  5. You are correct!
  6. Yessss!
  7. ?
  8. Flickr/Chris Voll
  9. Ego!
  10. Assuming != Determining
  11. Research!
  12. Agenda
  13. 9–10:30 Introduction. Collaboration and Research. Forming Your Questions. 10:30–11 Morning Tea Agenda 11–12:30 12:30–1:30 Activities. Interviewing. Lunch 1:30–3 Analysis & Models 3–3:30 Afternoon Tea 3:30–5 Analysis & Models Contd. Reporting & Sharing.Wrap-up
  14. Flickr/Jerome Collins
  15. Design-Led Design-led with participatory mindset Design-led with expert mindset Expert Mindset Users seen as subjects Participatory Mindset Users seen as partners Research-led with expert mindset Dubberly Design Office Research-led with participatory mindset Research-Led
  16. Goal Driven Increase chance of success Reduce risk Skeptical Mindset Willing to question the value of any approach
  17. Collaboration
  18. Dogma!
  19. How?
  20. Chris Noessel
  21. Why?
  22. Goals
  23. LeanUX Principles: Design thinking Agile methods Lean startup method
  24. Extreme Uncertainty
  25. Design Thinking
  26. “Design thinking is a humancentered approach to innovation that draws from the designer's toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.” – Tim Brown, president and CEO IDEO
  27. Dogma
  28. Context
  29. Real World Your Organization Your Users Context
  30. UX
  31. Business Business UX UX Business UX Business UX UX UX
  32. Locate your risk
  33. Flickr/Kris Krug
  34. Where are you coming from?
  35. Commit to working collaboratively
  36. Establish a process.
  37. Overcome objections
  38. We don’t have the time.
  39. We don’t have the money.
  40. We don’t have the expertise.
  41. We’re already A/B testing
  42. Everyone wants better products.
  43. No one wants to read a report.
  44. Design is active. Reading is passive. Research is active.
  45. Embrace conflict
  46. One Simple Process
  47. Form Questions Gather Data Analyze Data
  48. Form Questions Think Critically Analyze Data
  49. Form Questions Observe Analyze Data
  50. Form Questions Interview Analyze Data
  51. Form Questions Read Analyze Data
  52. Think Critically Observe Form Questions Interview Experiment Read Analyze Data
  53. Interpretation Interpretation Interpretation Interpretation
  54. Shared Reality
  55. Collaborating with Strangers
  56. A design project is a series of decisions.
  57. Research is a craft.
  58. Bias
  59. Confirmation Bias: You selectively weight the information that confirms what you already believe.
  60. Sampling Bias: Your sample of research subjects isn’t sufficiently representative.
  61. Interviewer Bias: You insert your opinion into interviews.
  62. Social Desirability Bias People don’t say the true things that they worry will make them look bad.
  63. Dunning-Kruger Effect Unskilled people feel overly confident. Competent people are less so.
  64. Forming Questions
  65. Good Questions Specific Actionable Practical
  66. A Bad Question “What do people think about pets?”
  67. A Better Question “How do single urban adults choose and acquire a pet?”
  68. A Bad Question “What do people do around here all day?”
  69. A Better Question “How do we coordinate communication priorities across departments?”
  70. Critical Thinking
  71. Critical Thinking Disciplined Self-correcting Clear Logical
  72. Uncritical Thinking “I hate yellow, so all yellow websites are total failures.”
  73. Critical Thinking “I hate yellow, but based on the evidence, it might work for our audience.”
  74. Research and Collaboration Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be. Done right, research and working collaboratively reinforce each other through a shared understanding of reality. Start with your goal in mind, not with any process or buzzword. Asking questions and cutting across traditional roles can both be threatening to the established order. Commit to clear communication and critical thinking. Research questions follow from goals, assumptions, and risk. Always have a framework and a plan.
  75. Break!
  76. Activities!
  77. Form Questions Gather Data Analyze Data
  78. Gather Data
  79. Interviews Interviews Users Contextual Inquiry Literature Review Usability Testing Descriptive Descriptive A/B Testing Evaluative Generative Questions About Org Analytic Product Evaluative Evaluative SWOT Analysis Usability Testing Analytic Competitive Analysis Brand Audit Competition Heuristic Analysis
  80. Research Activities Topic Purpose Time Money
  81. Purpose (Decision Type): What needs doing? What are people doing? How is this thing working?
  82. Purpose (Decision Type): Find new product idea. Better meet an identified need. Iterate on existing product.
  83. Purpose (Decision Type): Generative. Descriptive Evaluative
  84. Why Not Just Prototype?
  85. If we only test bottle openers, we may never realize customers prefer screw-top bottles. – Victor Lombardi, Why We Fail
  86. Organizational Research
  87. Organizational research is good for: Requirements Politics Workflow Capabilities Goodwill
  88. Requirements What are the top business priorities for this project/ product?
  89. Politics What does success mean to the individual stakeholders?
  90. Workflow Do we have to change how people are working together to be successful?
  91. Workflow How do we have to change how people are working together to be successful?
  92. Workflow How can we possibly change how people are working together?
  93. Capabilities What are the strengths and weaknesses of our team?
  94. Capabilities Where is the internal expertise?
  95. Goodwill How can this project make your job easier (or harder)?
  96. Basic Stakeholder Questions What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What does success mean from your perspective, what will have changed for the better once this project is complete? Do you have any concerns about this project? What do you think the greatest challenges to success are? Internal and external?
  97. For each stakeholder, note the following: What’s their general attitude toward this project? What’s the goal as they describe it? To what extent are this person’s incentives aligned with the project’s success? How much and what type of influence do they have? Who else do they communicate with on a regular basis? To what extent does this stakeholder need to participate throughout the project, and in which role? Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?
  98. Stakeholder power moves “Why are you asking me this?” “I don’t understand that question. It doesn’t make any sense.” “I don’t feel comfortable talking to you about that.” “No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.” “How much more time is this going to take?”
  99. 10 minutes practice. What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What do you think the greatest challenges to success are? Internal and external?
  100. User Research
  101. Ethnography Photo: Flickr/theloushe
  102. The Four Ds of Design Ethnography
  103. Deep Dive Daily Life Data Analysis Drama
  104. “...true ethnography reveals not just what people say they do, but what they actually do.” –PARC
  105. Photo: Flickr/lintmachine
  106. The Art of The Interview
  107. How to do bad user research: Ask what people want
  108. Everybody Lies
  109. Interviewing is not talking.
  110. Interviewing is listening.
  111. You The Comfort Zone Subject
  112. You The Comfort Zone Subject
  113. You The Comfort Zone Subject
  114. You The Comfort Zone Subject
  115. You The Comfort Zone Subject
  116. Good Interviewers: Know Your Question Warm Up Shut Up
  117. Introduction Body Conclusion
  118. Introduction: Smile Express gratitude Describe the process Ask to record Warm up questions
  119. Body: Ask open-ended questions Probe for more Allow silence Use questions as checklist
  120. Conclusion: Transition to wrap-up Ask if there is anything else Thank for time
  121. You are the host You are the student
  122. Interview Checklist Create a welcoming atmosphere to make participants feel at ease. Always listen more than you speak. Take responsibility to accurately convey the thoughts and behaviors of the people you are studying. Start each interview with a general description of the goal, but be careful of focusing responses too narrowly. Avoid leading questions and closed yes/no questions. Ask follow-up questions. Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it. Also note the exact phrases and vocabulary that participants use.
  123. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  124. Lunch!
  125. Interview Scenario You work for an e-Commerce site that wants to develop a new service to help people give gifts. The goal of the research is to identify unmet needs people might have with regard to giving gifts.
  126. Interview Practice Break into groups of 3-4 people 1 interviewee, interviewer , 1 notetaker, 1 observer (optional), Switch in 15 minutes 3 rounds
  127. Listen for: Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  128. How did that go?
  129. How about a focus group?
  130. “Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” –Robert K. Merton, Sociologist, the guy who invented focus groups
  131. Competitive Research
  132. How else might your target customer solve the same problem?
  133. Competitive Review How do they explicitly position themselves? What do they say they offer? Who do they appear to be targeting? How does this overlap or differ from your target audience or users? What are the key differentiators? The factors that make them uniquely valuable to their target market, if any? How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve? What do you notice that they’re doing particularly well or badly? Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?
  134. Your target customers have to love you more than they hate change.
  135. (Usability) Testing
  136. A good research activity: • Answers a key question • Addresses identified assumptions • Informs specific decisions • Involves your team • Fits your level of expertise • Fits your schedule and budget
  137. Collaborative Recruiting!
  138. How to find people: • From your existing, high-traffic site • Social networks • Friends and family • Mailing lists • Flyers
  139. A good research activity: • Answers a key question • Addresses identified assumptions • Informs specific decisions • Involves your team • Fits your level of expertise • Fits your schedule and budget
  140. • Fundamentally research is a simple process • There are many activities and definitions • No pressure! • Select the methods that inform decisions • Begin by understanding your organization • Never ask what people like • People are lazy, forgetful creatures of habit • Keep each other honest • Practice and learn
  141. Analysis and Models
  142. Creating Meaning From Data
  143. 1. Compile data 2. Analyze 3. Identify Insights 4. Create Model
  144. Analysis
  145. Basic Analysis Closely review the notes. Look for interesting behaviors, emotions, actions, and verbatim quotes. Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back). Group the notes. Watch the patterns emerge. Rearrange the notes as you continue to assess the patterns.
  146. Observation Observation Observation Observation Observation Observation
  147. Observation Observation Observation Observation Observation Observation Observation
  148. Collaborates on purchases Observation Observation Observation Observation Observation Observation Observation
  149. Collaborates on purchases Observation Observation Uses several devices Observation Observation Observation Observation Observation
  150. Collaborates on purchases Observation Observation Uses several devices Observation Observation Observation Needs affirmation Observation Observation
  151. Ground rules Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal. Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data. Clearly differentiate observations from interpretations (what happened versus what it means). No specific solutions until after you’ve gone through insights and principles. Solutions come next.
  152. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  153. 25 minutes analysis. Break into groups of 6-8 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.
  154. Models & Pictures
  155. Extract information
  156. Extract information
  157. Get thoughts out of your head
  158. Personas
  159. I’ve never seen a persona called “Married woman, no kids, with pristine hardwood.” God, how I aspire to see that persona. -Steve Portigal
  160. Make a persona based on your interviews Back into the analysis groups One person will describe the personas to everyone and we’ll decide whether they can be collapsed.
  161. Other Models
  162. ThoughtWorks
  163. A concept map is a picture of our understanding of something. –Dubberly Design Office
  164. Generate lists of words related to the main concept. The list can come from research, reading, experts, brainstorming, or any other source. The second step is to edit the list. Some terms may be related to the subject, but not in a way that meets the project goals. The third step is to define the terms on the edited list. This is particularly important with unfamiliar or technical terms. But it also helps with familiar terms, too. Create a matrix listing all the terms down one side and repeating the list across the top. Note the relationship in the boxes where a row and column intersect. The resulting matrix of relationships provides a checklist for building the concept map.
  165. • List terms • Edit the list • Define the remaining terms • Create a matrix showing the relations of terms • Rank the terms • Decide on main branches or write framing sentences • Fill in the rest of the structure • Revise • Apply typography to reinforce structure • Revise
  166. Analysis and Models Everyone on the team should be involved in turning data into insights. A productive session requires rules. Once you and your team have extracted insights from data, document those insights in models. A model distills and documents thinking so everyone on the team can see it and make decsions based upon it. Remember than models are still an interim document. They are tools. Think “useful” not precious. Update as needed. The affinity diagram comes straight out of analysis sessions. Personas are one of the most intelligible research outputs for people throughout the organization.
  167. Break!
  168. Reporting and Sharing
  169. How to make research meaningful to your organization
  170. Flickr/Kris Krug
  171. Flickr/Jerome Collins
  172. You are collaborating with your future selves.
  173. Design synthesis is the most critical part of the design process. Yet in our popular discussions of design and innovation, we've largely ignored this fundamental role. –John Kolko
  174. Building a Culture of Research
  175. How to make research meaningful to your organization
  176. It is your job to make it easy for everyone else.
  177. Study Title Date Completed Research Goal Related Decisions Activities Key Insights Supporting Observations Recommended Actions Questions for Further Study Research Report
  178. Clear goals Shared values Access to information Clear decision-making
  179. The goal determines the form
  180. How to apply research
  181. Reasons to Share
  182. Flickr/loozrboy
  183. The report is not the research.
  184. Why report at all?
  185. Informing? Inspiring? Focusing? Remembering? Recording? Deciding?
  186. Wrap Up
  187. In summary Research creates a shared understanding of reality. Asking questions is uncomfortable. Embrace that feeling. A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun. Clear goals and good questions are required. Choose only the research activities that answer real questions and inform your top priority design and development decisions. Practice! Observe and listen every day. Document! Report! Share! It’s easy to lose what you learn.
  188. Any questions?
  189. Brief books for people who make websites Erika Hall JUST ENOUGH RESEARCH 9 No. You might enjoy the book.