Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013


Published on

Silicon Valley Product Managers Association - Product Camp 2013

Published in: Design
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013

  1. 1. Top 10 User Research PitfallsDan ArraManaging Director
  2. 2. Agenda PART 1 About the Speaker PART 2 What is Research? PART 3 10 Research Pitfalls 2
  4. 4. About The Speaker: Dan Arra •  Began career as an engineer in the aerospace industry. •  Spent 10 years at software companies in systems engineering, product management and professional services. •  Started software & technology companies and was responsible for leading numerous successful projects. Another 10 years. •  Joined Macadamian, a software design & development studio. Work closely with design and software engineering teams to improve and create new products. 4
  6. 6. User-centered Design 6
  7. 7. UCD: understanding users and tasksGood design is built on a solid understanding of your audience: specific goals, tasks andcontext of use. Confidential 3/22/13 7
  8. 8. Research Methods: Qualitative versus Quantitative QUALITATIVE QUANTITATIVE attitude about Focus groups Surveys (online, telephone)your product and Interviews competition Behavior Ethnographic research A/B Testing related to your (Observation, shadowing) product offering usability of Usability testing Usability testing product Confidential 3/22/13 8
  10. 10. The most dangerous things about research! Myth:
 Being Smart means you can do good UX Research & Design. Anybody can ask a question… but relevant, appropriate, constructive? 10
  11. 11. Design research IS real science!Sci-ence [sahy-uh ns] noun•  systematic knowledge of the physical or material world gained through observation and experimentation. 11
  12. 12. Reality! Observe Think Design Test ! Rinse, repeat. 12
  13. 13. Our users don’t tell us what they needMyth:
We asked users what they wanted, so we recovered. 13
  14. 14. Users are NOT designers•  Users are focused on solutions•  Don’t consider context and other mitigating factors•  Why? Why? Why? 14
  15. 15. Avoid jumping to Solutions too quickly 15
  16. 16. Staying in problem space leads to INNOVATIONGo to: for full story. 16
  17. 17. Staying in problem space leads to INNOVATIONGo to: for full story. 17
  18. 18. Staying in problem space leads to INNOVATIONGo to: for full story. 18
  19. 19. Staying in problem space leads to INNOVATIONGo to: for full story. 19
  20. 20. Talking to customers " Research Who to ask... AND How to “ask”. 20
  21. 21. Consider Triangulating Research ! 21
  22. 22. Stating, without knowing! “We must have this feature”! can be very to EXPENSIVEGo to: for full story. 22
  23. 23. Numbers can be deceiving! 23
  24. 24. Sometimes numbers are not appropriate!! Qualitative Quantitative Attitude •  Focus Groups •  Surveys (Preference) •  Interviews •  A/B Testing Behavior •  Usability Walkthrough •  Usability Test (Performance) •  Ethnographic Research •  Benchmark Test •  Job Shadowing 24
  25. 25. Focus Groups & Usability Testing!NOT the same! What people say, whatpeople do, and what theysay they do are entirely different things. Photo credit: 25
  26. 26. Tale of Two methods Usability Testing Focus GroupsMethod One-on-one testing; observing representative Group discussion/exploration with up to samples of end users using the product and 10 participants per group measuring their performance on pre- (n=1) determined tasks (n= # of individuals being tested)When to use Throughout development phase and during Pre-development; early in program deployment verificationKey focus What will make the product easy for end What will motivate potential customers to users to learn and use? buy the product? Determining market need.Key How well can end users use the What are the customers wants and functionality a product provides? Can they desires ?motivation realize the value promised? 26
  27. 27. Eight is sometimes enough 27
  28. 28. It just dependsFaulkner, Laura (2003) Beyond the Five-user Assumption: Benefits of Increased Samplesizes in Usability Testing Behavior ResearchMethods, Instruments and Computers Volume 35 (3) 379-383 28
  29. 29. Raw data is NOT enough 29
  30. 30. Translate data into meaningful requirements 30
  31. 31. Don’t rely only on the Spec! 31
  32. 32. More than words can say! 32
  33. 33. Not just anyone can do research 33
  34. 34. UX Researcher – Prevents “heard what I wanted to hear.”•  Consider bringing someone on board with experience, a Degree in Human factors, HCI and/or soft sciences (e.g., Psychology, Sociology).•  If conducting true user research, you’ll want: −  someone with knowledge and experience in research methods and/or experimental practices. −  someone with the expertise to turn insights into design direction•  Partner user researcher with SMEs, business analysts, and product managers for maximum success! 34
  35. 35. A Good Researcher! !  can identify the best methods to get the right information !  is “familiar enough” with the subject material to determine the right questions !  avoids leading questions !  has strong listening skills !  follows the the natural flow of the conversation !  can steer the conversation in the case of digression and guide user to discuss new topics when one is exhausted !  remains neutral and puts the interviewee at ease !  focuses on the most recent experiences !  looks for ways to visualize and summarize results to be actionable 35
  36. 36. Remember Ask the right questions! Use the right method(s) Analyze appropriately Use the right people! 36
  37. 37. Remember, also that you are! Improving AND growing business. Developing relationships = gold. Establishing credibility = save time. Learning/refining your craft. 37
  38. 38. Thank-Youdarra@macadamian.comFor more information, visit: www.macadamian.comFollow Macadamian on Twitter: @macadamianlabs 38
  39. 39. Design and Research Resources”Innumeracy: Mathematical Illiteracy and Its Consequences”,by John Allen Paulos“Ambient Findability: What We Find Changes Who We Become”, by PeterMorville“Communicating Design: Developing Web Site Documentation for Designand Planning,” by Dan BrownThe Design of Everyday Things Don NormanDesigning Interfaces Jennifer TidwellDo You Matter? How Great Design Will Make People Love YourCompany Robert Brunner“Dont Make Me Think: A Common Sense Approach to Web Usability,” bySteve Krug Confidential 3/23/13 39
  40. 40. Design and Research Resources“The Elements of User Experience: User-Centered Design for the Web,”by Jesse James GarrettEmotional Design Don NormanMental Models: Aligning Design Strategy With Human Behavior by IndiYoung“Observing the User Experience: A Practitioner’s Guide to UserResearch,” by Mike Kuniavsky“Presentation Zen Garr ReynoldsSketching User Experiences: Getting the Design Right and the RightDesign,” by Bill BuxtonUniversal Principles of Design William Lidwell“Visual Explanations: Images and Quantities, Evidence and Narrative,” byEdward Tufte Confidential 3/22/13 40
  41. 41. About MacadamianWe create compelling user experiences for:•  Consumer and enterprise desktop applications•  Interfaces embedded into physical products•  Mobile interfaces and applications on multiple platforms•  Web applications and cloud-based applications Confidential 3/22/13 41
  42. 42. Design + Development!Confidential 3/22/13 42
  43. 43. •  Over 200 employees! •  15 + years in business! •  Integrated design & development teams! Ottawa! Yerevan!San Mateo! Cluj, Romania! Confidential 3/22/13 43
  44. 44. What We Do: UX services•  UX Research •  UX Design −  Stakeholder Interviews −  Information Architecture −  Product Strategy Workshops −  Content Modeling −  Task Analysis −  Interaction Design −  On-line Surveys −  Visual Design −  Usability Testing•  Software Development •  Software Quality Assurance-QA −  Stand alone or combined with design −  QA Review: performance or process −  PoC or market ready −  QA/QC for customer lead development −  Technologies: Web, Cloud, Mobile, −  QA/QC within our agile process Desktop −  Localization QA −  Agile methodology −  QA Script/Automation −  Integrated with customer teams 44