TI
AM Tutorial
10/1/2013 8:30:00 AM

"Exploring Usability Testing"
Presented by:
Rob Sabourin
AmiBug.com

Brought to you b...
Rob Sabourin
AmiBug.com
Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of
softwa...
Exploring Usability Testing
Robert Sabourin
President
AmiBug.Com, Inc.
Montreal, Canada
rsabourin@amibug.com
© 2013 Robert...
Exploring Usability Testing
• Robert Sabourin ,
Software Evangelist
• President
• AmiBug.Com Inc.
• Montreal, Quebec,
Cana...
Exploring Usability Testing

SOME PHILOSOPHY

5

© 2013 Robert Sabourin EUT v1.1

Conforming to
requirements
Suiting purpo...
Edsger W. Dijkstra
• “Program testing can be used to show the
presence of bugs, but never to show their
absence”

© 2013 R...
Non-Functional Testing
Quality Factors
Attributes
Characteristics
Reliability
Other “-ilities”
© 2013 Robert Sabourin EUT ...
Non-Functional Agile
Challenges
Time
•
•
•
•
•

Analysis
Model
Prepare
Run
Interpret

Change
• Baselines
• Goals

Software...
Quality Factors

13

© 2013 Robert Sabourin EUT v1.1

Product Backlog
Constraints
Tom Gilb, Competitive Engineering, sugge...
Non-Functional Testing

© 2013 Robert Sabourin EUT v1.1

15

Exploring Usability Testing

NET PROMOTER SCORE

© 2013 Rober...
Net Promoter Score

17

© 2013 Robert Sabourin EUT v1.1

Net Promoter Score

© 2013 Robert Sabourin EUT v1.1

© 2013 Rober...
Net Promoter Score

Response Scale
•11 values
•0 not at all likely
•10 extremely likely
19

© 2013 Robert Sabourin EUT v1....
Net Promoter Score
NPS
• The difference between the percentage
of Promoters and Detractors
• Is not expressed as a percent...
© 2013 Robert Sabourin EUT v1.1

23

Exploring Usability Testing

SYSTEM USABILITY SCALE

© 2013 Robert Sabourin EUT v1.1
...
System Usability Scale
SUS - System Usability Scale

• Measures usability in terms of
effectiveness, efficiency and satisf...
System Usability Scale
SUS Questions
1.
2.
3.
4.

I think that I would like to use this system frequently
I found the syst...
System Usability Scale
Scoring SUS

• For odd items: subtract one from the user
response.
• For even-numbered items: subtr...
Exploring Usability Testing

EXPLORATORY TESTING

31

© 2013 Robert Sabourin EUT v1.1

Exploratory Testing

Learning

Exec...
Charter Statement

•
•
•
•
•
•

Statement of mission
Ties to purpose
Focuses work
Confirms understanding
Delineates scope
...
eBay Usability Charters

© 2013 Robert Sabourin EUT v1.1

35

eBay Usability Charters

© 2013 Robert Sabourin EUT v1.1

© ...
eBay Usability Charters

© 2013 Robert Sabourin EUT v1.1

37

eBay Usability Charters

© 2013 Robert Sabourin EUT v1.1

© ...
Pairing Testers With …

Non Functional Testing
•Usability Gurus
•SMEs
•Human Factors
Experts
39

© 2013 Robert Sabourin EU...
Heuristics
Guide
Exploration
Rules of
thumb
Fallible but
useful

© 2013 Robert Sabourin EUT v1.1

41

Exploring Usability ...
Prototype Styles
•
•
•
•
•
•
•
•
•
•
•
•

Paper and pencil
Comic books style story boards
Wire frame models
Mock ups
Proto...
Whiteboarding

What is Whiteboarding?

Collaboration
Coordination
Conceptualization
45

© 2013 Robert Sabourin EUT v1.1

M...
Whiteboarding

47

© 2013 Robert Sabourin EUT v1.1

User Mind Map

Whiteboarding

© 2013 Robert Sabourin EUT v1.1

© 2013 ...
Buying a Book
Usage Scenarios

49

© 2013 Robert Sabourin EUT v1.1

Story Board

Whiteboarding

© 2013 Robert Sabourin EUT...
Wrap-O-Matic
Usage Scenarios

© 2013 Robert Sabourin EUT v1.1

51

Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1...
Buying a Book
Usage Scenarios

© 2013 Robert Sabourin EUT v1.1

53

Slide 54
© 2013 Robert Sabourin
EUT v1.1

© 2013 Rober...
Slide 55
© 2013 Robert Sabourin
EUT v1.1

© 2013 Robert Sabourin EUT v1.1

© 2013 Robert Sabourin

56

28
Social Networking
Usage Scenarios

© 2013 Robert Sabourin EUT v1.1

57

© 2013 Robert Sabourin EUT v1.1

58

© 2013 Robert...
Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

59

Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

© 20...
Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

61

Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

© 20...
Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

63

Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

© 20...
Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

65

Scenario Based Testing

© 2013 Robert Sabourin EUT v1.1

© 20...
TOFT Testing
• Task Oriented
Functional Testing
– Can the user
accomplish useful
tasks correctly?

© 2013 Robert Sabourin ...
Jakob Nielsen
• Usability heuristics
• Rules of thumb
• General principles
of user interface
design
• www.nngroup.com
69

...
Usability Heuristics

• Match between system and the real
world
– Speak the users' language
– Use familiar concepts
– Foll...
Usability Heuristics

• Consistency and standards
– Same words mean same thing in different
contexts
– Follow environment ...
Usability Heuristics

• Recognition rather than recall
– Making objects visible
– Make instructions available

75

© 2013 ...
Usability Heuristics

• Aesthetic and minimalist design

– Dialogues should contain
relevant information

77

© 2013 Rober...
Usability Heuristics

• Help and documentation
–
–
–
–

Easy to search
Focused on the user's task
Offer concrete steps to ...
Usability Severity

© 2013 Robert Sabourin EUT v1.1

81

Exploring Usability Testing

ISO 9421

© 2013 Robert Sabourin EUT...
Usability ISO 9421
Effectiveness
• the extent to which the intended goals of use are achieved

Efficiency
• the resources ...
Exploring Usability Testing

USABILITY TEST
FRAMEWORK
© 2013 Robert Sabourin EUT v1.1

85

Usability Test Framework
Basic ...
Usability Test Framework

User groups
• Select 5 or 6 users for each group
• All should attempt the same tasks
• Complex a...
Usability Test Framework
Prepare Usability Scenario
• Identify project usability test objectives
• Select relevant user ta...
Usability Test Framework
Think Aloud
Approach

Retrospective
Think Aloud

• the user
verbalizes actions
and decisions
• us...
Usability Test Framework
Measures and metrics
•Effectiveness
•Efficiency
•Satisfaction
•Error frequency
•Memorability
93

...
Usability Test Framework
Facilitating a test
• Make participant comfortable with environment and
technology
• Give partici...
Form Design Principles

•
•
•
•

Minimize pain
Illuminate path
Consider context
Ensure consistent communication
Luke Wrobl...
Form Usability

•
•
•
•
•
•

Number and location of errors
Severity of errors
Completion rates
Time to complete forms
Sati...
Clear Form Path

101

© 2013 Robert Sabourin EUT v1.1

Form Usability

• Eye tracking
– What people looked at
– Number of ...
Form Usability Checklist

Any unnecessary questions
Succinct questions
Sincere voice
Natural language
Logical groups ...
Progress Indication

105

© 2013 Robert Sabourin EUT v1.1

Input Validation

Suggest valid answers
Illustrate valid form...
Error Messages

Provide clear error messages
Indicate if error blocks completion
Guide users to resolve error
Provide ...
Do it yourself

109

© 2013 Robert Sabourin EUT v1.1

Do it yourself
A morning a month
Get insights from users
Watch real ...
Do it yourself
Recruit diversely
Find important problems
Improve design
111

© 2013 Robert Sabourin EUT v1.1

Do it yourse...
Do it yourself
Observers take notes
User questions recorded
Action decided at lunch
© 2013 Robert Sabourin EUT v1.1

113

...
Mobile Usability

Difficulties
•Small Screens
•Limited bandwidth
•Fingers
•No Flash Like GUI
115

© 2013 Robert Sabourin E...
Mobile Usability
Guidelines
•
•
•
•
•

Important stuff on top
Easy to find stuff
Single column layout
Minimize navigation
...
Mobile Usability

Mobile Usability Test
•In the lab
•Beta
•Crowd source
119

© 2013 Robert Sabourin EUT v1.1

Mobile Usabi...
Mobile Usability
Vary and control
• Geographically
• Continent
• Country
• City
• Language
• Locale
121

© 2013 Robert Sab...
Exploring Usability Testing

LEAN UX

123

© 2013 Robert Sabourin EUT v1.1

Lean UX

© 2013 Robert Sabourin EUT v1.1

© 20...
Lean UX
Lean Principles
• Cross functional teams
• Small teams
• Dedicated teams
• Collocated teams
• Progress equals Outc...
Lean UX
Lean Principles
• Externalize Your Work
• Making over Analysis
• Learning over Growth
• Permission to Fail
• Getti...
Lean UX

Assumptions
•Business
•User
129

© 2013 Robert Sabourin EUT v1.1

Lean UX

Hypotheses

© 2013 Robert Sabourin EUT...
Lean UX

Outcomes
More business
More referrals

More fun
131

© 2013 Robert Sabourin EUT v1.1

Lean UX

Personas
• Sketch ...
Lean UX

Features are
•Capabilities
•Tactics
•Products
133

© 2013 Robert Sabourin EUT v1.1

Lean UX

Features serve

•Bus...
Lean UX

Features drive
customer
behaviour
135

© 2013 Robert Sabourin EUT v1.1

Lean UX

Process

© 2013 Robert Sabourin ...
Thank You
• Questions?

© 2013 Robert Sabourin EUT v1.1

© 2013 Robert Sabourin

137

69
Upcoming SlideShare
Loading in...5
×

Exploring Usability Testing

234
-1

Published on

It is not enough to verify that software conforms to requirements by passing established acceptance tests. Successful software products engage, entertain, and support the users' experience. While goals vary from project to project, no matter how robust and reliable your software is, if your users do not embrace it, business can slip from your hands. Rob Sabourin shares how to elicit effective usability requirements with techniques such as story boarding and task analysis. Together, testers, programmers, and users collaborate to blend the requirement, design, and test cycles into a tight feedback loop. Learn how to select a subset of system functions to test with a small group of users to get high value information at low cost. Learn how usability testers can take advantage of naïve questions from novice users as well as the tunnel vision and bias of domain experts. Rob shares examples of usability testing for a variety of technologies including mobile and web based products.

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
234
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Exploring Usability Testing

  1. 1. TI AM Tutorial 10/1/2013 8:30:00 AM "Exploring Usability Testing" Presented by: Rob Sabourin AmiBug.com Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Rob Sabourin AmiBug.com Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software development professionals. A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization.
  3. 3. Exploring Usability Testing Robert Sabourin President AmiBug.Com, Inc. Montreal, Canada rsabourin@amibug.com © 2013 Robert Sabourin EUT v1.1 1 Exploring Usability Testing WELCOME © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 2 1
  4. 4. Exploring Usability Testing • Robert Sabourin , Software Evangelist • President • AmiBug.Com Inc. • Montreal, Quebec, Canada • rsabourin@amibug.com • www.amibugshare.com 3 © 2013 Robert Sabourin EUT v1.1 Exploring Usability Testing • Pain points? – What hurts? – How Much? © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 4 2
  5. 5. Exploring Usability Testing SOME PHILOSOPHY 5 © 2013 Robert Sabourin EUT v1.1 Conforming to requirements Suiting purpose Delivering value to stakeholders © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 6 3
  6. 6. Edsger W. Dijkstra • “Program testing can be used to show the presence of bugs, but never to show their absence” © 2013 Robert Sabourin EUT v1.1 7 Exploring Usability Testing NON-FUNCTIONAL TESTING © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 8 4
  7. 7. Non-Functional Testing Quality Factors Attributes Characteristics Reliability Other “-ilities” © 2013 Robert Sabourin EUT v1.1 9 Non-Functional Testing Challenges Goals • Elicit • Articulate • Quantify Subjective • Needs or wants? • Relative or absolute? • What is good enough? Tests • Tricky to orchestrate • Difficult to interpret results • Challenging to baseline or regress © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 10 5
  8. 8. Non-Functional Agile Challenges Time • • • • • Analysis Model Prepare Run Interpret Change • Baselines • Goals Software • Continuous integration • Incomplete product builds 11 © 2013 Robert Sabourin EUT v1.1 ISO 9126 Quality Factors Usability © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin • Attributes of the effort needed for use, and on the individual assessment of such use, by a set of users • Learnability • Understandibility • Operability 12 6
  9. 9. Quality Factors 13 © 2013 Robert Sabourin EUT v1.1 Product Backlog Constraints Tom Gilb, Competitive Engineering, suggests defining Scale: "What is measured" Meter: "How to measure (method)" Target: "Level we're aiming for. Success" Constraint: "Level we're seeking to avoid. Failure" Benchmark: "Where we are today" © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 14 7
  10. 10. Non-Functional Testing © 2013 Robert Sabourin EUT v1.1 15 Exploring Usability Testing NET PROMOTER SCORE © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 16 8
  11. 11. Net Promoter Score 17 © 2013 Robert Sabourin EUT v1.1 Net Promoter Score © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 18 9
  12. 12. Net Promoter Score Response Scale •11 values •0 not at all likely •10 extremely likely 19 © 2013 Robert Sabourin EUT v1.1 Net Promoter Score Promoters • respondents giving a 9 or 10 score Passives • respondents giving a 7 or 8 score Detractors • respondents giving a 0 to 6 score © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 20 10
  13. 13. Net Promoter Score NPS • The difference between the percentage of Promoters and Detractors • Is not expressed as a percentage • Is an absolute number lying between -100 and +100 • uTest suggests “any positive number is good” (ref: TK Maxx Mobile Usability Testing Results Report) 21 © 2013 Robert Sabourin EUT v1.1 Net Promoter Score © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 22 11
  14. 14. © 2013 Robert Sabourin EUT v1.1 23 Exploring Usability Testing SYSTEM USABILITY SCALE © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 24 12
  15. 15. System Usability Scale SUS - System Usability Scale • Measures usability in terms of effectiveness, efficiency and satisfaction • SUS can be read as a percentage • SUS can be used as a benchmarking mechanism • SUS can be used to compare the current system to competitors’ systems • SUS can be used to compare the current system to future versions © 2013 Robert Sabourin EUT v1.1 25 System Usability Scale © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 26 13
  16. 16. System Usability Scale SUS Questions 1. 2. 3. 4. I think that I would like to use this system frequently I found the system unnecessarily complex I thought the system was easy to use I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system 27 © 2013 Robert Sabourin EUT v1.1 System Usability Scale SUS Responses © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 28 14
  17. 17. System Usability Scale Scoring SUS • For odd items: subtract one from the user response. • For even-numbered items: subtract the user responses from 5 • This scales all values from 0 to 4 (with four being the most positive response). • Add up the converted responses for each user and multiply that total by 2.5. This converts the range of possible values from 0 to 100 instead of from 0 to 40. 29 © 2013 Robert Sabourin EUT v1.1 System Usability Scale Scoring SUS © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 30 15
  18. 18. Exploring Usability Testing EXPLORATORY TESTING 31 © 2013 Robert Sabourin EUT v1.1 Exploratory Testing Learning Execution © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin Design 32 16
  19. 19. Charter Statement • • • • • • Statement of mission Ties to purpose Focuses work Confirms understanding Delineates scope Analogy to test story 33 © 2013 Robert Sabourin EUT v1.1 Charter Statement • • • • • Short, terse To the point Inclusions … Exclusions … Limits … © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 34 17
  20. 20. eBay Usability Charters © 2013 Robert Sabourin EUT v1.1 35 eBay Usability Charters © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 36 18
  21. 21. eBay Usability Charters © 2013 Robert Sabourin EUT v1.1 37 eBay Usability Charters © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 38 19
  22. 22. Pairing Testers With … Non Functional Testing •Usability Gurus •SMEs •Human Factors Experts 39 © 2013 Robert Sabourin EUT v1.1 Heuristics © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 40 20
  23. 23. Heuristics Guide Exploration Rules of thumb Fallible but useful © 2013 Robert Sabourin EUT v1.1 41 Exploring Usability Testing WHITEBOARDING © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 42 21
  24. 24. Prototype Styles • • • • • • • • • • • • Paper and pencil Comic books style story boards Wire frame models Mock ups Prototypes Site maps User flow Wire flow Story boards Wire frame Page description diagram Functional specification 43 © 2013 Robert Sabourin EUT v1.1 Whiteboarding © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 44 22
  25. 25. Whiteboarding What is Whiteboarding? Collaboration Coordination Conceptualization 45 © 2013 Robert Sabourin EUT v1.1 Mind Maps Tony Buzan © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 46 23
  26. 26. Whiteboarding 47 © 2013 Robert Sabourin EUT v1.1 User Mind Map Whiteboarding © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 48 24
  27. 27. Buying a Book Usage Scenarios 49 © 2013 Robert Sabourin EUT v1.1 Story Board Whiteboarding © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 50 25
  28. 28. Wrap-O-Matic Usage Scenarios © 2013 Robert Sabourin EUT v1.1 51 Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 52 26
  29. 29. Buying a Book Usage Scenarios © 2013 Robert Sabourin EUT v1.1 53 Slide 54 © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 27
  30. 30. Slide 55 © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 56 28
  31. 31. Social Networking Usage Scenarios © 2013 Robert Sabourin EUT v1.1 57 © 2013 Robert Sabourin EUT v1.1 58 © 2013 Robert Sabourin 29
  32. 32. Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 59 Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 60 30
  33. 33. Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 61 Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 62 31
  34. 34. Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 63 Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 64 32
  35. 35. Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 65 Scenario Based Testing © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 66 33
  36. 36. TOFT Testing • Task Oriented Functional Testing – Can the user accomplish useful tasks correctly? © 2013 Robert Sabourin EUT v1.1 Slide 67 Exploring Usability Testing USABILITY HEURISTICS © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 68 34
  37. 37. Jakob Nielsen • Usability heuristics • Rules of thumb • General principles of user interface design • www.nngroup.com 69 © 2013 Robert Sabourin EUT v1.1 Usability Heuristics • Visibility of system status – Always keep users informed – Provide appropriate feedback – Respond in a reasonable timeframe © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 70 35
  38. 38. Usability Heuristics • Match between system and the real world – Speak the users' language – Use familiar concepts – Follow a natural order 71 © 2013 Robert Sabourin EUT v1.1 Usability Heuristics • User control and freedom – Provide means to exit from unwanted states – Provide undo and redo capabilities © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 72 36
  39. 39. Usability Heuristics • Consistency and standards – Same words mean same thing in different contexts – Follow environment conventions 73 © 2013 Robert Sabourin EUT v1.1 Usability Heuristics • Error prevention – Prevents problems from occurring – Eliminate error-prone conditions – Have a confirmation option © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 74 37
  40. 40. Usability Heuristics • Recognition rather than recall – Making objects visible – Make instructions available 75 © 2013 Robert Sabourin EUT v1.1 Usability Heuristics • Flexibility and efficiency of use – Accelerators for the expert user – Make common options adaptable © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 76 38
  41. 41. Usability Heuristics • Aesthetic and minimalist design – Dialogues should contain relevant information 77 © 2013 Robert Sabourin EUT v1.1 Usability Heuristics • Help users recognize, diagnose, and recover from errors – Use plain language for error messages – Constructively suggest solutions © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 78 39
  42. 42. Usability Heuristics • Help and documentation – – – – Easy to search Focused on the user's task Offer concrete steps to be carried out Short and to the point © 2013 Robert Sabourin EUT v1.1 79 Exploring Usability Testing USABILITY SEVERITY LEVELS © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 80 40
  43. 43. Usability Severity © 2013 Robert Sabourin EUT v1.1 81 Exploring Usability Testing ISO 9421 © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 82 41
  44. 44. Usability ISO 9421 Effectiveness • the extent to which the intended goals of use are achieved Efficiency • the resources that have been expended to achieve the intended goals Satisfaction • the extent to which the user finds the use of the product acceptable Safety • can user be harmed 83 © 2013 Robert Sabourin EUT v1.1 Usability ISO 9421 Completeness • the extent to which the system is complete in design and meets its desired goals Learn-ability • Is the capability of a software product to enable the user to learn how to use it Convenience • the extent to which the product fits into normal course of daily life Ambiguity • Confusing or vague usability. Misleading information that might also result in providing no usability at all © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 84 42
  45. 45. Exploring Usability Testing USABILITY TEST FRAMEWORK © 2013 Robert Sabourin EUT v1.1 85 Usability Test Framework Basic Usability Test Steps • Select representative users • Complete defined set of tasks • Facilitator observes user behaviour • Interview user after task is completed • User is asked to think aloud (recorded) • Review findings to identify concerns • Generate actionable recommendations © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 86 43
  46. 46. Usability Test Framework User groups • Select 5 or 6 users for each group • All should attempt the same tasks • Complex applications may need many groups • Groups can be based • Domain (subject matter) expertise • Experience level (task or professional) • Solution familiarity © 2013 Robert Sabourin EUT v1.1 87 Usability Test Framework Where to perform usability tests? • Control lab • Conference room with recording equipment • Work area with • Local • Distributed with local observer • Distributed without local observer © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 88 44
  47. 47. Usability Test Framework Prepare Usability Scenario • Identify project usability test objectives • Select relevant user tasks • Keep user instructions short and to the point • Do not over complicate instructions • Use the language of the user not the language of the system • Make task parameters specific • Use a logical flow 89 © 2013 Robert Sabourin EUT v1.1 Usability Test Framework Running a test •Let participants make mistakes •Take note of questions •Take note of decisions points •Take note of alternative selections © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 90 45
  48. 48. Usability Test Framework Think Aloud Approach Retrospective Think Aloud • the user verbalizes actions and decisions • user verbalizes questions concerns or ambiguities • records and eye tracking software record event • review with user and ask them to explain their actions 91 © 2013 Robert Sabourin EUT v1.1 Usability Test Framework Reporting Findings • • • • List of usability concerns Summary of user questions Description of users behaviour Tasks of concern • Too long • Awkward • Unsuccessful • Time to complete • Feedback • Expected • Unexpected © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 92 46
  49. 49. Usability Test Framework Measures and metrics •Effectiveness •Efficiency •Satisfaction •Error frequency •Memorability 93 © 2013 Robert Sabourin EUT v1.1 Usability Test Framework Measures and metrics • Successful task completion • Error rates • Time on task • Subjective comments • Likes • Dislikes • Recommendations © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 94 47
  50. 50. Usability Test Framework Facilitating a test • Make participant comfortable with environment and technology • Give participants a change to ask questions • Explain process • Ask participants about the amount of experience they have with the technology • Ask participants about the amount of experience they have with the domain • Review task description with participant • Explain facilitators involvement © 2013 Robert Sabourin EUT v1.1 95 Exploring Usability Testing FORMS AND WORKFLOWS © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 96 48
  51. 51. Form Design Principles • • • • Minimize pain Illuminate path Consider context Ensure consistent communication Luke Wroblewski Web Form Design 97 © 2013 Robert Sabourin EUT v1.1 Eye Tracking © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 98 49
  52. 52. Form Usability • • • • • • Number and location of errors Severity of errors Completion rates Time to complete forms Satisfaction scores Subjective comments 99 © 2013 Robert Sabourin EUT v1.1 Awkward Form Path © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 100 50
  53. 53. Clear Form Path 101 © 2013 Robert Sabourin EUT v1.1 Form Usability • Eye tracking – What people looked at – Number of eye fixations – Length of eye fixations © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 102 51
  54. 54. Form Usability Checklist Any unnecessary questions Succinct questions Sincere voice Natural language Logical groups of information Structure form as a conversation Ask optional questions separately © 2013 Robert Sabourin EUT v1.1 103 Form Usability Checklist Name the form Name sections of the form Clear scan line Spacing between sections Minimal distractions Start page guidance for long forms Indicate progress © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 104 52
  55. 55. Progress Indication 105 © 2013 Robert Sabourin EUT v1.1 Input Validation Suggest valid answers Illustrate valid format Validate syntax after user enters data Indicate limits or ranges of inputs Provide smart defaults Hide unneeded controls © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 106 53
  56. 56. Error Messages Provide clear error messages Indicate if error blocks completion Guide users to resolve error Provide visual emphasis of error Use red text and icons for errors Indicate successful completion © 2013 Robert Sabourin EUT v1.1 107 Exploring Usability Testing DO IT YOURSELF USABILITY TESTING © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 108 54
  57. 57. Do it yourself 109 © 2013 Robert Sabourin EUT v1.1 Do it yourself A morning a month Get insights from users Watch real users © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 110 55
  58. 58. Do it yourself Recruit diversely Find important problems Improve design 111 © 2013 Robert Sabourin EUT v1.1 Do it yourself Three testers Test on site Observe shared screens © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 112 56
  59. 59. Do it yourself Observers take notes User questions recorded Action decided at lunch © 2013 Robert Sabourin EUT v1.1 113 Exploring Usability Testing SOME MOBILE USABILITY CONCERNS © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 114 57
  60. 60. Mobile Usability Difficulties •Small Screens •Limited bandwidth •Fingers •No Flash Like GUI 115 © 2013 Robert Sabourin EUT v1.1 Mobile Usability Risks •Bad first impression •Operator errors •Negative feedback / reviews •Competitor fills gap © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 116 58
  61. 61. Mobile Usability Guidelines • • • • • Important stuff on top Easy to find stuff Single column layout Minimize navigation Light data 117 © 2013 Robert Sabourin EUT v1.1 Mobile Usability Guidelines • • • • • • Finger taps Avoid fancy web gui technologies Minimize forms Minimize feature sets Separate mobile web resources Fast © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 118 59
  62. 62. Mobile Usability Mobile Usability Test •In the lab •Beta •Crowd source 119 © 2013 Robert Sabourin EUT v1.1 Mobile Usability Mobile Usability Test • Vary and control • Technologically • OS • Browser • Device • Carrier • Bandwidth © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 120 60
  63. 63. Mobile Usability Vary and control • Geographically • Continent • Country • City • Language • Locale 121 © 2013 Robert Sabourin EUT v1.1 Mobile Usability Vary and control • Demographically • Age • Gender • Education • Employment • Industry • Computer savvy • Domain expertise © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 122 61
  64. 64. Exploring Usability Testing LEAN UX 123 © 2013 Robert Sabourin EUT v1.1 Lean UX © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 124 62
  65. 65. Lean UX Lean Principles • Cross functional teams • Small teams • Dedicated teams • Collocated teams • Progress equals Outcomes • Problem focused teams 125 © 2013 Robert Sabourin EUT v1.1 Lean UX Lean Principles • Remove waste • Small batch size • Continuous discovery • GOOB Get out of the building • Shared understanding • No Rock Stars Gurus or Ninjas © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 126 63
  66. 66. Lean UX Lean Principles • Externalize Your Work • Making over Analysis • Learning over Growth • Permission to Fail • Getting out of the Deliverables Business 127 © 2013 Robert Sabourin EUT v1.1 Lean UX Assumptions Hypotheses Outcomes Personas Features © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 128 64
  67. 67. Lean UX Assumptions •Business •User 129 © 2013 Robert Sabourin EUT v1.1 Lean UX Hypotheses © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin We believe that creating this experience for these personas will achieve this outcome. We will know this to be true when we see this feedback or measure. 130 65
  68. 68. Lean UX Outcomes More business More referrals More fun 131 © 2013 Robert Sabourin EUT v1.1 Lean UX Personas • Sketch and Name • Demographic • Pain points & needs • Potential solutions © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 132 66
  69. 69. Lean UX Features are •Capabilities •Tactics •Products 133 © 2013 Robert Sabourin EUT v1.1 Lean UX Features serve •Business •Customer •User © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 134 67
  70. 70. Lean UX Features drive customer behaviour 135 © 2013 Robert Sabourin EUT v1.1 Lean UX Process © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 136 68
  71. 71. Thank You • Questions? © 2013 Robert Sabourin EUT v1.1 © 2013 Robert Sabourin 137 69

×