Your SlideShare is downloading. ×
Collaborative Research
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Collaborative Research


Published on

Slides from my Collaborative Research workshop from Webstock 2014. …

Slides from my Collaborative Research workshop from Webstock 2014.

Buy the book: Just Enough Research

Published in: Design, Technology, Education
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Collaborative Research with Erika Hall
  • 2. Hello!
  • 3. I have a question...
  • 4. Do you enjoy being right?
  • 5. You are correct!
  • 6. Yessss!
  • 7. ?
  • 8. Flickr/Chris Voll
  • 9. Ego!
  • 10. Assuming != Determining
  • 11. Research!
  • 12. Agenda
  • 13. 9–10:30 Introduction. Collaboration and Research. Forming Your Questions. 10:30–11 Morning Tea Agenda 11–12:30 12:30–1:30 Activities. Interviewing. Lunch 1:30–3 Analysis & Models 3–3:30 Afternoon Tea 3:30–5 Analysis & Models Contd. Reporting & Sharing.Wrap-up
  • 14. Flickr/Jerome Collins
  • 15. Design-Led Design-led with participatory mindset Design-led with expert mindset Expert Mindset Users seen as subjects Participatory Mindset Users seen as partners Research-led with expert mindset Dubberly Design Office Research-led with participatory mindset Research-Led
  • 16. Goal Driven Increase chance of success Reduce risk Skeptical Mindset Willing to question the value of any approach
  • 17. Collaboration
  • 18. Dogma!
  • 19. How?
  • 20. Chris Noessel
  • 21. Why?
  • 22. Goals
  • 23. LeanUX Principles: Design thinking Agile methods Lean startup method
  • 24. Extreme Uncertainty
  • 25. Design Thinking
  • 26. “Design thinking is a humancentered approach to innovation that draws from the designer's toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.” – Tim Brown, president and CEO IDEO
  • 27. Dogma
  • 28. Context
  • 29. Real World Your Organization Your Users Context
  • 30. UX
  • 31. Business Business UX UX Business UX Business UX UX UX
  • 32. Locate your risk
  • 33. Flickr/Kris Krug
  • 34. Where are you coming from?
  • 35. Commit to working collaboratively
  • 36. Establish a process.
  • 37. Overcome objections
  • 38. We don’t have the time.
  • 39. We don’t have the money.
  • 40. We don’t have the expertise.
  • 41. We’re already A/B testing
  • 42. Everyone wants better products.
  • 43. No one wants to read a report.
  • 44. Design is active. Reading is passive. Research is active.
  • 45. Embrace conflict
  • 46. One Simple Process
  • 47. Form Questions Gather Data Analyze Data
  • 48. Form Questions Think Critically Analyze Data
  • 49. Form Questions Observe Analyze Data
  • 50. Form Questions Interview Analyze Data
  • 51. Form Questions Read Analyze Data
  • 52. Think Critically Observe Form Questions Interview Experiment Read Analyze Data
  • 53. Interpretation Interpretation Interpretation Interpretation
  • 54. Shared Reality
  • 55. Collaborating with Strangers
  • 56. A design project is a series of decisions.
  • 57. Research is a craft.
  • 58. Bias
  • 59. Confirmation Bias: You selectively weight the information that confirms what you already believe.
  • 60. Sampling Bias: Your sample of research subjects isn’t sufficiently representative.
  • 61. Interviewer Bias: You insert your opinion into interviews.
  • 62. Social Desirability Bias People don’t say the true things that they worry will make them look bad.
  • 63. Dunning-Kruger Effect Unskilled people feel overly confident. Competent people are less so.
  • 64. Forming Questions
  • 65. Good Questions Specific Actionable Practical
  • 66. A Bad Question “What do people think about pets?”
  • 67. A Better Question “How do single urban adults choose and acquire a pet?”
  • 68. A Bad Question “What do people do around here all day?”
  • 69. A Better Question “How do we coordinate communication priorities across departments?”
  • 70. Critical Thinking
  • 71. Critical Thinking Disciplined Self-correcting Clear Logical
  • 72. Uncritical Thinking “I hate yellow, so all yellow websites are total failures.”
  • 73. Critical Thinking “I hate yellow, but based on the evidence, it might work for our audience.”
  • 74. Research and Collaboration Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be. Done right, research and working collaboratively reinforce each other through a shared understanding of reality. Start with your goal in mind, not with any process or buzzword. Asking questions and cutting across traditional roles can both be threatening to the established order. Commit to clear communication and critical thinking. Research questions follow from goals, assumptions, and risk. Always have a framework and a plan.
  • 75. Break!
  • 76. Activities!
  • 77. Form Questions Gather Data Analyze Data
  • 78. Gather Data
  • 79. Interviews Interviews Users Contextual Inquiry Literature Review Usability Testing Descriptive Descriptive A/B Testing Evaluative Generative Questions About Org Analytic Product Evaluative Evaluative SWOT Analysis Usability Testing Analytic Competitive Analysis Brand Audit Competition Heuristic Analysis
  • 80. Research Activities Topic Purpose Time Money
  • 81. Purpose (Decision Type): What needs doing? What are people doing? How is this thing working?
  • 82. Purpose (Decision Type): Find new product idea. Better meet an identified need. Iterate on existing product.
  • 83. Purpose (Decision Type): Generative. Descriptive Evaluative
  • 84. Why Not Just Prototype?
  • 85. If we only test bottle openers, we may never realize customers prefer screw-top bottles. – Victor Lombardi, Why We Fail
  • 86. Organizational Research
  • 87. Organizational research is good for: Requirements Politics Workflow Capabilities Goodwill
  • 88. Requirements What are the top business priorities for this project/ product?
  • 89. Politics What does success mean to the individual stakeholders?
  • 90. Workflow Do we have to change how people are working together to be successful?
  • 91. Workflow How do we have to change how people are working together to be successful?
  • 92. Workflow How can we possibly change how people are working together?
  • 93. Capabilities What are the strengths and weaknesses of our team?
  • 94. Capabilities Where is the internal expertise?
  • 95. Goodwill How can this project make your job easier (or harder)?
  • 96. Basic Stakeholder Questions What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What does success mean from your perspective, what will have changed for the better once this project is complete? Do you have any concerns about this project? What do you think the greatest challenges to success are? Internal and external?
  • 97. For each stakeholder, note the following: What’s their general attitude toward this project? What’s the goal as they describe it? To what extent are this person’s incentives aligned with the project’s success? How much and what type of influence do they have? Who else do they communicate with on a regular basis? To what extent does this stakeholder need to participate throughout the project, and in which role? Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?
  • 98. Stakeholder power moves “Why are you asking me this?” “I don’t understand that question. It doesn’t make any sense.” “I don’t feel comfortable talking to you about that.” “No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.” “How much more time is this going to take?”
  • 99. 10 minutes practice. What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What do you think the greatest challenges to success are? Internal and external?
  • 100. User Research
  • 101. Ethnography Photo: Flickr/theloushe
  • 102. The Four Ds of Design Ethnography
  • 103. Deep Dive Daily Life Data Analysis Drama
  • 104. “...true ethnography reveals not just what people say they do, but what they actually do.” –PARC
  • 105. Photo: Flickr/lintmachine
  • 106. The Art of The Interview
  • 107. How to do bad user research: Ask what people want
  • 108. Everybody Lies
  • 109. Interviewing is not talking.
  • 110. Interviewing is listening.
  • 111. You The Comfort Zone Subject
  • 112. You The Comfort Zone Subject
  • 113. You The Comfort Zone Subject
  • 114. You The Comfort Zone Subject
  • 115. You The Comfort Zone Subject
  • 116. Good Interviewers: Know Your Question Warm Up Shut Up
  • 117. Introduction Body Conclusion
  • 118. Introduction: Smile Express gratitude Describe the process Ask to record Warm up questions
  • 119. Body: Ask open-ended questions Probe for more Allow silence Use questions as checklist
  • 120. Conclusion: Transition to wrap-up Ask if there is anything else Thank for time
  • 121. You are the host You are the student
  • 122. Interview Checklist Create a welcoming atmosphere to make participants feel at ease. Always listen more than you speak. Take responsibility to accurately convey the thoughts and behaviors of the people you are studying. Start each interview with a general description of the goal, but be careful of focusing responses too narrowly. Avoid leading questions and closed yes/no questions. Ask follow-up questions. Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it. Also note the exact phrases and vocabulary that participants use.
  • 123. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  • 124. Lunch!
  • 125. Interview Scenario You work for an e-Commerce site that wants to develop a new service to help people give gifts. The goal of the research is to identify unmet needs people might have with regard to giving gifts.
  • 126. Interview Practice Break into groups of 3-4 people 1 interviewee, interviewer , 1 notetaker, 1 observer (optional), Switch in 15 minutes 3 rounds
  • 127. Listen for: Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  • 128. How did that go?
  • 129. How about a focus group?
  • 130. “Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” –Robert K. Merton, Sociologist, the guy who invented focus groups
  • 131. Competitive Research
  • 132. How else might your target customer solve the same problem?
  • 133. Competitive Review How do they explicitly position themselves? What do they say they offer? Who do they appear to be targeting? How does this overlap or differ from your target audience or users? What are the key differentiators? The factors that make them uniquely valuable to their target market, if any? How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve? What do you notice that they’re doing particularly well or badly? Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?
  • 134. Your target customers have to love you more than they hate change.
  • 135. (Usability) Testing
  • 136. A good research activity: • Answers a key question • Addresses identified assumptions • Informs specific decisions • Involves your team • Fits your level of expertise • Fits your schedule and budget
  • 137. Collaborative Recruiting!
  • 138. How to find people: • From your existing, high-traffic site • Social networks • Friends and family • Mailing lists • Flyers
  • 139. A good research activity: • Answers a key question • Addresses identified assumptions • Informs specific decisions • Involves your team • Fits your level of expertise • Fits your schedule and budget
  • 140. • Fundamentally research is a simple process • There are many activities and definitions • No pressure! • Select the methods that inform decisions • Begin by understanding your organization • Never ask what people like • People are lazy, forgetful creatures of habit • Keep each other honest • Practice and learn
  • 141. Analysis and Models
  • 142. Creating Meaning From Data
  • 143. 1. Compile data 2. Analyze 3. Identify Insights 4. Create Model
  • 144. Analysis
  • 145. Basic Analysis Closely review the notes. Look for interesting behaviors, emotions, actions, and verbatim quotes. Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back). Group the notes. Watch the patterns emerge. Rearrange the notes as you continue to assess the patterns.
  • 146. Observation Observation Observation Observation Observation Observation
  • 147. Observation Observation Observation Observation Observation Observation Observation
  • 148. Collaborates on purchases Observation Observation Observation Observation Observation Observation Observation
  • 149. Collaborates on purchases Observation Observation Uses several devices Observation Observation Observation Observation Observation
  • 150. Collaborates on purchases Observation Observation Uses several devices Observation Observation Observation Needs affirmation Observation Observation
  • 151. Ground rules Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal. Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data. Clearly differentiate observations from interpretations (what happened versus what it means). No specific solutions until after you’ve gone through insights and principles. Solutions come next.
  • 152. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  • 153. 25 minutes analysis. Break into groups of 6-8 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.
  • 154. Models & Pictures
  • 155. Extract information
  • 156. Extract information
  • 157. Get thoughts out of your head
  • 158. Personas
  • 159. I’ve never seen a persona called “Married woman, no kids, with pristine hardwood.” God, how I aspire to see that persona. -Steve Portigal
  • 160. Make a persona based on your interviews Back into the analysis groups One person will describe the personas to everyone and we’ll decide whether they can be collapsed.
  • 161. Other Models
  • 162. ThoughtWorks
  • 163. A concept map is a picture of our understanding of something. –Dubberly Design Office
  • 164. Generate lists of words related to the main concept. The list can come from research, reading, experts, brainstorming, or any other source. The second step is to edit the list. Some terms may be related to the subject, but not in a way that meets the project goals. The third step is to define the terms on the edited list. This is particularly important with unfamiliar or technical terms. But it also helps with familiar terms, too. Create a matrix listing all the terms down one side and repeating the list across the top. Note the relationship in the boxes where a row and column intersect. The resulting matrix of relationships provides a checklist for building the concept map.
  • 165. • List terms • Edit the list • Define the remaining terms • Create a matrix showing the relations of terms • Rank the terms • Decide on main branches or write framing sentences • Fill in the rest of the structure • Revise • Apply typography to reinforce structure • Revise
  • 166. Analysis and Models Everyone on the team should be involved in turning data into insights. A productive session requires rules. Once you and your team have extracted insights from data, document those insights in models. A model distills and documents thinking so everyone on the team can see it and make decsions based upon it. Remember than models are still an interim document. They are tools. Think “useful” not precious. Update as needed. The affinity diagram comes straight out of analysis sessions. Personas are one of the most intelligible research outputs for people throughout the organization.
  • 167. Break!
  • 168. Reporting and Sharing
  • 169. How to make research meaningful to your organization
  • 170. Flickr/Kris Krug
  • 171. Flickr/Jerome Collins
  • 172. You are collaborating with your future selves.
  • 173. Design synthesis is the most critical part of the design process. Yet in our popular discussions of design and innovation, we've largely ignored this fundamental role. –John Kolko
  • 174. Building a Culture of Research
  • 175. How to make research meaningful to your organization
  • 176. It is your job to make it easy for everyone else.
  • 177. Study Title Date Completed Research Goal Related Decisions Activities Key Insights Supporting Observations Recommended Actions Questions for Further Study Research Report
  • 178. Clear goals Shared values Access to information Clear decision-making
  • 179. The goal determines the form
  • 180. How to apply research
  • 181. Reasons to Share
  • 182. Flickr/loozrboy
  • 183. The report is not the research.
  • 184. Why report at all?
  • 185. Informing? Inspiring? Focusing? Remembering? Recording? Deciding?
  • 186. Wrap Up
  • 187. In summary Research creates a shared understanding of reality. Asking questions is uncomfortable. Embrace that feeling. A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun. Clear goals and good questions are required. Choose only the research activities that answer real questions and inform your top priority design and development decisions. Practice! Observe and listen every day. Document! Report! Share! It’s easy to lose what you learn.
  • 188. Any questions?
  • 189. Brief books for people who make websites Erika Hall JUST ENOUGH RESEARCH 9 No. You might enjoy the book.