#flashtest: User Research Live


Published on

Kirk Doggett and Kate Lawrence's presentation from the UXPA Boston 2014 Conference.

Published in: Design, Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

#flashtest: User Research Live

  1. 1. Kirk Doggett Manager, Global UX Research, Vistaprint @GR8UX #FLASHTEST: User Research Live UXPA Boston, 2014 Kate Lawrence Director, User Research, EBSCO Information Services @bykatelawrence Kirk
  2. 2. What you will learn today about the usertesting.com tool: We’ll launch a test, and while it’s running, tell you how Vistaprint and EBSCO structure their tests and report on the findings. One size does not fit all; we’ll tell you when it is appropriate to use a tool like usertesting.com and when you are better off with another method. Why un-moderated testing tools should be a part of your research toolkit, and why Usertesting.com works for Vistaprint and EBSCO. Why we like it When to use it How to use it
  3. 3. 3, 2, 1.…. Launch!
  4. 4. • Single largest player in the internet printing market, helping small businesses around the world create professional marketing products at an affordable price • 25 localized websites • 17 million customers • 3 state of the art manufacturing plants • 13 offices worldwide and growing Kirk
  5. 5. User Research at Vistaprint How: • In-person testing • Remote testing • UT.com • Phone Surveys • Field Research • Customer Council • M-Turk What: • Our site • Competitors • Prototypes Kirk
  6. 6. • Premier provider of research databases, eJournals and eBooks worldwide • Powerful discovery tools & management resources for libraries and other institutions • Point-of-care medical resources and corporate learning tools • We’re hiring! www.ebscohost.com/careers Ipswich location, electric car charging stations & river views Kate
  7. 7. User Research at EBSCO Primary Research: Asking the users ourselves Data/ Usage Analysis: What do the metrics show? Review of Secondary Research: What did others learn? Kate
  8. 8. Broad Selection of Tools Kirk and Kate +
  9. 9. What we like about Usertesting.com  It’s fast – results are typically back within 1 hour.  You can recruit specific users – gender, age, country, income, web expertise, other (must be a small business owner, must be a college student, etc)  You can provide a URL where you want users to start – can be a site, dropbox, sharefile, etc.  Test anything – powerpoints, prototypes, sites (yours, competitors)  Give users 15 minutes worth of work  Have 4 free-text questions after the tasks are complete. Kate
  10. 10. What We’re Testing Today
  11. 11. Test Details  Website URL: http://www.vistaprint.com/business-cards.aspx  Scenario: Imagine you need to get business cards to represent yourself and your profession.  Tasks: 1. Familiarize yourself with this page, and mention your first impressions. 2. Click the “Browse our designs” section to continue. 3. Look carefully to find a business card that appeals to you and represents you/your business; Find one you would *really* want to buy. 4. After finding the design you like best, stop at the point where you begin to customize the card. Comment on the process of finding a design.  Questions for Participants: 1. What options or tools did you notice to help find your design? Was there anything missing? 2. What was the best part about finding a design?” 3. What frustrated you most about the process of finding a design? 4. What is the one most important improvement this site could make to make it easier to find a design?  Participant Demographics  Gender – 50% M; 50% F  30 to 60 years of age  Country: US  Income range from $40K+  Web expertise: average web user  One other such as “must be a small business owner”
  12. 12. Power Tip: The Pilot Test • Use a single session to test the clarity of your instructions • How well does the panelist respond? • Are you learning what you set out to learn? • If the pilot works, then run the rest of the sample • If not, modify the test, and pilot again
  13. 13. Vistaprint: One-Week Test Plan Monday • Kickoff • Goals • Expectations • Budget Tuesday • Starting URL • Scenario • Tasks • Questions • Demographics Wednesday • Refine plan • Sign-off • Staging site • Prototype, etc. Thursday • Pilot • Launch • Analysis • Begin report Friday • Report results UX Research Business Stakeholders
  14. 14. Kate Usertesting.com for Branding Skoodle Perooz Nimbus Iris Snap Right Now Breeze Bloom Boundless Launch Rendezvous
  15. 15. Flexible Anytime Anywhere Holidays Evenings Weekends No travel Benefits
  16. 16. Can also be used for Mobile Phones & Tablets Kirk
  17. 17. • Quick & easy • Great for targeted tasks (“surgical strike”) • A great value • For desktop or mobile • Test anything from wireframes to competitor sites • Videos play in the browser • Efficient - playback 1.5X or 2X • Videos are downloadable • Clips and annotations • Tasks must be very clear – remember, users are working on their own • Users have to actually follow directions • Un-moderated: No way to course-correct • Not for time-consuming, complex tasks • Selection bias: higher expertise user base • You must watch all the videos Usertesting.com – When To Use It Kirk and Kate
  18. 18. Video Answers User stats (technographics) Video annotations Video clips Downloadable videos spreadsheet Review Live Results
  19. 19. Questions?
  20. 20. Thank you kdoggett@vistaprint.com klawrence@ebsco.com @GR8UX @bykatelawrence