The document discusses usability testing as a way to validate products for lean startups. It provides an overview of usability and users, explains how to conduct usability testing, and discusses how usability testing fits into the lean startup process. Examples of guerrilla usability testing techniques are also presented. The document encourages testing with a small number of users and running many small tests to improve products iteratively.

1. Lean Startup Baltimore
Usability Testing as Validation
Amy Rubino November 26, 2012
@Amy_Rubino

2. About Me
Amy Rubino
User Experience Consultant, Systems
Alliance
• 15 years of design experience as a
print and web designer, information
architect and user experience
advocate.
arubino@systemsalliance.com • Worked with universities,
linkedin.com/pub/amy-rubino/13/837/31b corporations and non-profits.
@amy_rubino
• Master’s degree in Interaction Design
and Information Architecture.
Usability Testing as Validation 2

3. What We’ll Cover
Usability
Users
Learn About Users
Learn From Users
Usability Testing
How to Do It
What You Can Test
Why You Should Do It
Usability Testing and Lean Startup
Demo
Resources
Photo credit: Amy Rubino
Usability Testing as Validation 3

4. Usability
Photo credit: http://www.flickr.com/photos/reticulating 4

5. User Experience Honeycomb
Photo credit: http://semanticstudios.com/
Usability Testing as Validation 5

6. Usability
 Creating products that are easy to use
Require less mental effort to use
Decrease frequency of mistakes and mistakes are less disastrous
Can be used to do more or do it faster
Can be figured out quicker
Usability Testing as Validation 6

7. Usability
Photo credit: http://impossibleobjects.com
Usability Testing as Validation 7

8. Usability
 Based on human behavior
 Focus on people who will use
it or users
Photo credit: sanjaykumarux.blogspot.com
Usability Testing as Validation 8

9. Users = Real people
Use your website or app
Part of your target
audience
Not directly affiliated with
your company
Photo credit: http://www.leveltendesign.com/ 9

10. You Are NOT Like Your Users
You Your Users
Know a lot about Know little or nothing about
your product and company your product or company
Are passionate about Just want to get things
what you offer done
Have ideas on how your May use your website or
website or app will be used app differently than was
based on being an insider intended
Usability Testing as Validation 10

11. Learn About Users
Surveys
Focus groups
Web analytics
Photo credit: http://www.flickr.com/photos/scottandsarah 11

12. Learn From Users
Talk to them
Observe them in their own
environment
Usability testing
Photo credit: http://www.flickr.com/photos/blesseurope 12

13. Usability Testing
Photo credit: http://www.openroad.ca 13

14. “ … If you want to know whether your software…
is easy enough to use, watch some people while
they try to use it and note where they run into
trouble. Then fix it, and test it again.”
Steve Krug
Author, “Don’t Make Me Think” and “Rocket
Surgery Made Easy”
Usability Testing as Validation 14

15. Traditional Process
Develop a test plan
Choose a testing environment
Find and select participants
Prepare test materials
Conduct the sessions
Debrief with participants and
observers
Analyze data and observations
Create findings and
recommendations
Photo credit: http://www.flickr.com/photos/doos/3050195096 15

16. Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing 16

17. Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing 17 17

18. Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing 18 18

19. Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing 19 19

20. In-Person Usability Testing
 Invite people to see a beta version and
watch them complete tasks
 Pros
See and hear participants’ reactions
Can record conversation and click path
Your stakeholders and team can
participate
 Cons
You or participants have to travel to meet
up
Photo credit: http://www.flickr.com/photos/curiouslee/
May not have access to participants who
live outside of this region
Usability Testing as Validation 20

21. Remote Usability Testing
 Use screen sharing software to
watch participants complete tasks
in another location
 Pros
Don’t have to travel to participants
Have access to participants who live
outside of this region
Can record conversation and click
path
Your stakeholders and team can
participate
 Cons Photo credit: http://www.flickr.com/photos/rosenfeldmedia/
Don’t see participants’ reactions
Usability Testing as Validation 21

22. Unmoderated Usability Testing
 Use a service that allows participants to complete the tasks
on their one time without you moderating
 Pros
Participants can do it when it’s convenient for them
Have access to participants who live outside of this region
 Cons
Can’t ask follow up questions
Don’t hear participants think out loud
Don’t see participants’ reactions
Usability Testing as Validation 22

23. What You Can Test
Real products
Digital prototypes
Paper prototypes
Photo credit: http://www.jamesjpierce.com/energy_challenge_08.html 23

24. DO
 Do test the site NOT the participants
 Do note participant’s success in “I would open
completing each task Goggle and search
 Do ask open-ended follow up for the website.”
questions Quote from usability testing
participant
 Do compensate participants
Usability Testing as Validation 24

25. DON’T
 Don’t give participants tasks that
“Do you really can not be completed
think people will  Don’t guide participants
use this?”  Don’t answer questions until the
Quote from usability testing
end
participant
 Don’t take it personally
Usability Testing as Validation 25

26. Why You Should Do It
 Reduce costs Fix Errors
Once a piece of software $120
makes it into the field, $100
the cost of fixing an error
$80
can be 100 times as
high as it would have $60
been during the $40
development stage.
$20
Source:
$0
IEEE (Institute of Electrical and Electronics During After
Engineers)
http://spectrum.ieee.org/computing/software/
Development Completion
why-software-fails/
Usability Testing as Validation 26

27. Why You Should Do It
 Increase revenue
Increase sales, applications, memberships or donations
Increase retention
Remain competitive
Usability Testing as Validation 27

28. Why You Should Do It
 Improve effectiveness
Reduce user error
Increase efficiency
Increase user satisfaction
Photo credit: http://www.flickr.com/photos/terryfreedman
Usability Testing as Validation 28

29. Usability Testing and Lean Startup
Photo credit: http://thebln.com 29

30. Lean Startup
Photo credit: http://lean.st/
Usability Testing as Validation 30

31. Traditional Process
Develop a test plan
Choose a testing environment
Find and select participants
Prepare test materials
Conduct the sessions
Debrief with participants and
observers
Analyze data and observations
Create findings and
recommendations
Photo credit: http://www.flickr.com/photos/doos/3050195096 31

32. Lean Process
Develop a test plan
Choose a testing environment
Prepare test materials
Find and select participants
Conduct the sessions
Debrief with participants and
observers
Analyze data and observations
Create findings and
recommendations
Photo credit: http://www.flickr.com/photos/adactio/ 32

33. Guerilla Usability Testing
 Go to a public place and ask customers
to complete tasks
 Pros
See and hear participants’ reactions
See participants use product on different
devices
 Cons
Might need to get manager’s approval
Customers may not be your target
audience
Your stakeholders and team can’t Photo credit: http://www.flickr.com/photos/_dchri
participate
Usability Testing as Validation 33

34. “ … The best results come from testing no more
than 5 users and running as many small tests as
you can afford.”
Jakob Nielsen
principal, Nielsen Norman Group
Usability Testing as Validation 34

35. Demo
Photo credit: http://www.flickr.com/photos/vaniercollege 35

36. Try it Yourself!
 Join Baltimore Lean Startup on Facebook:
https://www.facebook.com/groups/BaltimoreLeanStartup
 Do some guerrilla testing
 Post your results
Usability Testing as Validation 36

37. Resources
Photo credit: http://www.flickr.com/photos/rodrigogalindez/ 37

38. Books
 “Don’t Make Me Think: A Common
Sense Approach to Web Usability”
by Steve Krug
 “Handbook of Usability Testing:
How to Plan, Design, and Conduct
Effective Tests”
by Jeffrey Rubin and Dana Chisnell
 “Remote Research: Real Users,
Real Time, Real Research” Photo credit: amazon.com
by Nate Bolt and Tony
Tulathimutte
Usability Testing as Validation 38

39. Websites
 sensible.com
 useit.com
 usability.gov
 alistapart.com
 uie.com
Photo credit: http://usability.gov/
Usability Testing as Validation 39

40. Testing Tools
 Morae
 Optimal Workshop
 UserTesting.com
 Loop11
 GoToMeeting
 Skype
Photo credit: http://beta.skype.com/en/
Usability Testing as Validation 40

41. Questions?
Photo credit: http://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you%27re_a_dog
Usability Testing as Validation 41

42. Contact
Amy Rubino
arubino@systemsalliance.com
Stay in touch:
www.systemsalliance.com
@systemsalliance
info@systemsalliance.com
Usability Testing as Validation 42