Rating high severity issues as 1 or 0 is confusing. The human brain thinks that higher numbers deserve more attention. This is a legacy of the Engineering teams these models were build for
Notice the way the Wilson scale focuses on data loss not actual user impact
We deal with the User, not the system
The Molich system focuses heavily on time on task, not frustration
Sauro’s model covers user impact and time, but does not focus on the business impact of the issue. Is the task important?
These are key road arteries in London where parking, stopping, or standing is not allowed AT ANY TIME. This system improved bus journeys to be 10% faster and 27% more reliable 6% reduction in accidents This same concept applies to web sites/applications in terms of identifying critical actions and high frequency actions Paying a bill on your site - frequent and critical Key business objectives Key customer objectives
Complete activities that may involve several interactions, pages, etc Concrete definition of “success”
Does the issue occur on a “red route”? These are critical user and/or business tasks in the interface This is where your business partners get to “be a part of the process” in helping identify the red routes, and if the task occurred here Was the task difficult to overcome? Did the study participant ask for help? Did they say they would give up at this point? Did they exhibit visual signs of frustration? Was the problem persistent? Did this occur multiple times to a single participant? Did this occur for at least 50% of your participants?
Critical - The highest urgency level of issues where users are unable to complete red route tasks or give up High - The issue causes a significant slow down in a user’s task completion time Medium - The issues causes moderate levels of frustration but the user was able to complete the task Low - These issues often are aesthetic in nature and don’t cause frustration or block task completion. They are a poor reflection of the product or brand
99 Problem, But is this issue one?
Who am I?
• AJ Siegel
• UX Consultant @ ServiceNow
• Twitter: @stegel36
• Translating findings into action
• Remember, our goal is a better UI
• Where do we even begin?
“Originality is nothing but judicious imitation. The
most original writers borrowed one from another.”
These don’t work…for me
So we find an issue…
“I saw 3 of the 5 sessions, and this one guy
got really mad when he couldn’t find the
Submit button, this issue is totally a High
“After facilitating all 5 sessions, I really only
recall 1 person expressing frustration, lets
call this a medium”
• Jakob Nielsen
• 0 - I don’t agree that this is a usability problem at all
• 4 - Usability catastrophe: imperative to fix this
before product can be released
• Jeff Rubin
• 4 - Unusable: The user is not able to or will not want
to use a particular part of the product because of the
way that the product has been designed and
• 1 - Irritant: The problem occurs only intermittently,
can be circumvented easily, or is dependent on a
standard that is outside the product's boundaries.
Could also be a cosmetic problem.
• Joe Dumas & Ginny Redish
• 1 - Prevents Task Completion
• 4 - Subtle and possible enhancements/suggestions
• Chauncey Wilson
• 1 - Catastrophic error causing irrevocable
loss of data or damage…
• 5 - Minimal error. The problem is rare and
causes no data loss…
• Molich & Jeffries
• 1 - Minor: delays user briefly.
• 3 - Catastrophic: prevents user from
completing their task.
• Jeff Sauro
• 1 - Minor: Causes some hesitation or slight
• 3 - Critical: Leads to task failure. Causes
user extreme irritation.
Can we make it less arbitrary?
Let’s talk about London’s buses
How do we use this concept for UX?
• Identify most important tasks
• Track issue frequency
• Observe difficulty to overcome