Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NoVA UX Meetup: Product Testing and Data-informed Design

6,436 views

Published on

These are the slides for the January 2013 NoVA UX Meetup in Vienna, VA. VP of UX Jim Lane shared tips, tools, and research strategies that the AddThis has used to develop publisher products used on over 14 million websites.

Published in: Design

NoVA UX Meetup: Product Testing and Data-informed Design

  1. Product Testing and Data-informed UX
  2. Introductions
  3. Who the heck are you?Jim LaneVP, UX AddThisOrganizer, NoVA UX Meetupjim@addthis.com@jimlanenova
  4. Ok, whats AddThis?AddThis a data, distribution and decisioningcompany delivering powerful, easy to useconsumer engagement tools and services.We provide: ● Consumer engagement tools ● Audience and data products ● Library of APIs and SDKs ● Browser extensions
  5. By the numbers...2008 2012● 50 services ● 330 services● 17 languages ● 70 languages● 116k domains ● 14M domains● 300M views daily ● 3B+ views daily ● 10TB of data daily
  6. 2009 Presidential Inauguration = 1.5M people 3B views = 2000 inaugurations The little things matter...
  7. Product testing and data-informed UX
  8. Three key takeawaysResearch + data + testing = informed designBeing informed is easier than you think.No excuses!
  9. A caveat...Data is important, but there are otherconsiderations as well... ● brand identity ● business objectives ● context (ex: mobile)Lets get started with...
  10. User-centered Design Approach Plan Observe Prototype Design Test Goals Surveys Paper Wireframes Quick tests Metrics Field research Interactive PDF Design mocks A/B testsBaselines User groups HTML Specifications Multi-variate Personas User testing Diagrams
  11. Planning
  12. Defining things up frontStart by understanding who your audience is: ● Who are they? ● What are they trying to do? What are their pain points? ● What problems are they trying to solve? ● Who are primary vs secondary personas?At AddThis, we have a lot of different kinds of users, soutilizing multiple personas helps us structure different kindsof design problems for different groups of people.
  13. Google AnalyticsUnderstand the existing state ● Basic site metrics ● Navigation analysis ● Demographics ● What are the metrics that really matter? ● Baseline before experimentingAdvanced segments can be used to apply historicalanalysis and side-by-side comparison to visits that share aparticular characteristic, such as users of a particular OS.
  14. Tip: Advanced Segments in GA
  15. Observing
  16. Enlist your users!Lots of ways to engage your audience ● Social media channels (Twitter, Facebook) ● Email feedback tools ● Support SaaS (Assistly) ● Stakeholder interviews (Skype interviews) ● Start a user group (Google Hangouts) ● Surveys (SurveyMonkey)Tip: SurveyMonkey crosstabs can help identify trends bycross-referencing answers to poll questions based onanswers to other questions, such as company size or levelof experience.
  17. General survey Intro survey ● ● ● Constructing a User Group
  18. Tip: Remote Card Sorts with Google Docs
  19. Prototyping & Design
  20. Fast and iterativeMinimum viable design fosters speed anditerative usability enhancement ● White boards ● Paper ● Omnigraffle ○ diagram -> wireframe -> design mock -> assets ● Balsamiq, other fast prototype-to-design-to-code solutions
  21. Using Post-Its to Solicit Feedback from Peers
  22. Testing
  23. TestingStart with understanding the hypothesisyoure trying to evaluate. Be sure youresolving the right problem!Plan for testing as part of the design anddevelopment process.Statistical significance and confidenceintervals matter.
  24. Testing hypothesesStrategies ● A/B or multi-cell to get the big picture ● Multi-variate to fine-tune ● Funnel analysis to optimize conversions ● Release and measure for complex interactions or little stuffTools ● GA & Google Content Experiments (formerly Google Website Optimizer) for multi-cell ● Optimiz.ly or Convert.com for multi-cell and multi-variate ● In-house A/B testing
  25. User testingMethods ● Task completion rate ● Task completion time ● Cognitive models and process analysis ● Track usability longitudinallyTools ● 5-second test ● Usertesting.com
  26. Dont forget about monitoring● Google Analytics● Product & brand scorecards (longitudinal)● Internal reporting and alerting
  27. Lets recap
  28. User-centered Design Approach Plan Observe Prototype Design Test Goals Surveys Paper Wireframes Quick tests Metrics Field research Interactive PDF Design mocks A/B testsBaselines User groups HTML Specifications Multi-variate Personas User testing Diagrams (whether youre waterfall or agile!)
  29. Lots of strategies to choose from...Quantitative Qualitative● Google Analytics ● User surveys● CrazyEgg.com ● Email feedback● 5secondtest.com ● Assistly● PDF prototypes ● User groups● A/B & multi-variate testing ● Google+ hangouts● User testing ● Social comments are data● Facebook Insights ● ... and some of these can● TwitterCounter.com et al. be quantified!● Internal analytics systems● Some can also provide qualitative data
  30. Three key takeawaysResearch + data + testing = informed designBeing informed is easier than you think.No excuses!
  31. Resources● Quantifying the User Experience (Sauro/Lewis)● Advanced Web Metrics with Google Analytics (Clifton)● Web Analytics An Hour a Day (Kaushik)● HeadFirst Statistics (Griffiths)● Landing Page Optimization (Ash)
  32. Thank you!Jim LaneVP, UX AddThisOrganizer, NoVA UX Meetupjim@addthis.com@jimlanenova

×