I led a team of three of MHCID students in using Neilsen's ten usability heuristics to evaluate Say Insurance's website. A heuristic evaluation can help user experience researchers and designers uncover most usability issues without user testing.

1. H E U R I S T I C E V A L U A T I O N
1.
INTRO

2. Positive findings
Say Insurance's new Insurance Score feature
meets many usability criteria: it lets the user
know how many steps are in the form; provides
content that speaks the user's, not the system's,
language; offers accelerators for expert,
returning or mobile users; and has a clean
aesthetic.
Negative findings
This report found room for improvement on user control,
consistency in the menu, and implementation of error
prevention. It is interesting to note that some evaluators
encountered initial difficulties in distinguishing the
insurance score and the insurance quote features on
the home page. This report includes recommendations
to improve this issue by using interface design that
supports recognition rather than recall.
2.
EXECUTIVE SUMMARY
Only one feature was ranked
severe: Users must give personal
information before learning
they're not able to get their
Insurance Score.

3. •• MARIA HAYNIE
AMIR RASHIDIFAR
DAVIDSON YOUNG
SHIRIN DIVINPOUR
SAY INSURANCE
PROJECT
OWNER
EVALUATORS
SUBJECT
3.
TEAM INTRODUCTION

4. WHAT IS SAY
INSURANCE?
Say is a young, online-only auto insurance provider backed by a
traditional agent-model insurance company headquartered in
Missouri.
4.
CLIENT INTRODUCTION

5. WHAT IS THE
INSURANCE SCORE?
In late 2017, Say released a new feature called Insurance Score.
Similar to a credit score but evaluated from an auto insurance
perspective, the Insurance Score allows a user can find out the
factors that impact their car insurance rates and learn what they
can do about it.
5.
FEATURE INTRODUCTION

6. METHODOLOGY
For this project, we studied the following evaluation methodologies:
Peter Morville's User Experience Honeycomb
Ben Shneiderman's Eight Golden Rules of Interface Design
Albert Lund's Expert Rating of Usability Maxims
Jakob Nielsen's Ten Usability Heuristics for Interface Design
I decided on Nielsen's criteria because it encompasses many of the concerns
the other addressand is a well-recognized industry standard.
6.
METHODOLOGY

7. HOW TO USE THESE
FINDINGS
The purpose of this evaluation was to evaluate the Insurance Score feature
with Nielsen's ten heuristics in order to improve the user experience on the Say
Insurance site.
Negative findings were ranked on a scale of 0-4, zero being a minor usability
problem and four being a severe usability problem. Positive findings were not
ranked.
For each negative finding, a recommendation is provided to improve the
design.
7.
METHODOLOGY

8. Visit SayInsurance.com Fill out address,
apartment number,
email and last
four of your social
security number.
Read intro and select "get score."
Select "insurance score"
from the top menu or "Tell
me more" from lower on
the home page.
See your Insurance Score
FIll out first name,
last name and
date of birth text
input fields, then
select next.
TASK OVERVIEW: EVALUATE THE "INSURANCE SCORE" FEATURE...
1 2 3
45
6
8.
TASK OVERVIEW

9. H1
Visibility of system status
Keep users informed
about what is going on
through appropriate
feedback within
reasonable time.
H2
Match between system
and the real world
Speak the users'
language, follow real-
world conventions and
use natural and logical
ordering.
H3
User control and
freedom
Users often make
mistakes and need
a clearly marked
"emergency exit" without
extended dialogue.
Support undo and redo.
H4
Consistency and
standards
Users should not have
to wonder whether
different words,
situations, or actions
mean the same thing.
H5
Error prevention
Either eliminate error-
prone conditions or
check for them and
present users with a
confirmation option
beforehand.
H6
Recognition rather than
recall
Minimize the user's
memory load by
making objects, actions,
and options visible
with easily retrievable
instructions.
H7
Flexibility and efficiency
of use
Use accelerators to speed
up the interaction for the
expert user and allow
users to tailor frequent
actions.
H8
Aesthetic and
minimalist design
Every extra unit of
information in a dialogue
competes with the
relevant units and
diminishes their relative
visibility.
H9
Help users recognize,
diagnose, and recover
from errors
Error messages
should be expressed
in plain language
(no codes), precisely
indicate the problem,
and suggest a
solution.
H10
Help and documentation
Any necessary help
and documentation
should be easy to search,
focused on the user's
task, list concrete steps
to be carried out, and not
be too large.
...USING NIELSEN'S TEN HEURISTICS.
9.
NIELSEN'S TEN
HEURISTICS

10. HEURISTIC POSITIVE NEGATIVE
H1
H2
H3 X
H4 X
H5 X
H6 X
H7
H8 X
H9
H10
POSITIVE NEGATIVE
7 5
TOTAL FINDINGS
FINDINGS SUMMARY
10.
FINDINGS SUMMARY

11. Positive Findings
11.
POSITIVE FINDINGS

12. H1: Visibility of system status
KEEP THE USERS INFORMED ABOUT WHAT IS GOING ON
H1.1 The introductory Insurance Score screen lets the user know how long the score should take.
H1.2 The first form screen in the Insurance Score flow lets the user know how many steps are left
in the process.
H1.2
H1.1
12.
POSITIVE FINDINGS

13. H2: Match between system
and the real world
SPEAK THE USER'S LANGUAGE
H2.1 The content uses words and phrases familiar to the user.
H2.1
13.
POSITIVE FINDINGS

14. H7: Flexibility and
efficiency of use
USE ACCELERATORS TO SPEED UP
INTERACTIONS
H7.1 The returning user can retrieve a saved quote.
H7.2 The address selection includes a predictive dropdown,
which makes it easier and faster for all users, especially
those on mobile.
ALLOW USERS TO TAILOR FREQUENT
ACTIONS
H7.3 On mobile, the phone's native autofill behavior is
allowed for easier and faster form-filling.
H7.1
H7.2
14.
POSITIVE FINDINGS

15. H8: Aesthetic and minimalist
design
CLEAN DESIGN
H8.1 The home page is not overly busy, there is some visual hierarchy of information, and there is
a clear call-to-action.
H8.1
15.
POSITIVE FINDINGS

16. H9: Help users
recognize, diagnose,
and recover from errors
ERROR MESSAGES SHOULD BE EXPRESSED
IN PLAIN LANGUAGE AND SUGGEST A SOLUTION
H9.1 Error messages are clear and easy to understand. The error
message changes when a date of birth is mistyped versus when
the field is left blank. H9.1H9.1
16.
POSITIVE FINDINGS

17. H10: Help and documentation
PROVIDE EASY-TO-ACCESS HELP
H10.1 On every page, there is a tap-to-call link to customer service above the main menu and a
chat feature available that floats on the right lower corner as the user navigates throughout the
site.
H10.1
H10.1
17.
POSITIVE FINDINGS

18. Negative Findings
18.
NEGATIVE FINDINGS

19. H3: User control and freedom
X ALLOW USERS TO LEAVE AN UNWANTED STATE WITHOUT HAVING TO
GO THROUGH AN EXTENDED DIALOGUE
H3.1 The user who inputs all of the fields accurately is disappointed to reach the end only to find
that they are not able to achieve their goal and obtain an Insurance Score because they are out of
service areas. This was the only item ranked as a severe usability problem.
Consider letting users know earlier in the Insurance Score process whether they are in service
areas or not. Consider telling the user before they have to provide any personal information
or consider a state field in the address section with a drop-down that only includes the states
serviced by Say.
H3.1
SEVERE
19.
NEGATIVE FINDINGS

20. H4: Consistency and
standards
X FOLLOW PLATFORM CONVENTIONS
H4.1 The main menu changes between the home page and the secondary level pages of the
insurance score. The color changes and the secondary menu disappears. While this minimizes the
chance the user will move out of the Insurance Score, it is inconsistent.
Consider maintaining consistency in the main menu's color and behavior.
H4.1
The menu changes from
the home page on the
insurance score page.
MILD
20.
NEGATIVE FINDINGS

21. H5: Error prevention
X PREVENT A PROBLEM FROM OCCURING IN THE FIRST PLACE
H5.1 Errors are not prevented. Errors show up after the users submit their information. Required
fields are not indicated, and hint text inside the fields are not provided.
Consider stopping users from being able to select the "next" button to go on to the next page until
all inputs have been filled in correctly (see example from Lemonade's quote below. The button
can not be selected until there is text in the fields).
H5.2
H5.1
MILD
21.
NEGATIVE FINDINGS

22. H6: Recognition rather than
recall
X THE USER SHOULD NOT HAVE TO REMEMBER INFORMATION FROM
ONE PART OF THE DIALOGUE TO ANOTHER.
H6.1 Once the user starts the Insurance Score form, nothing on the screen tells the user where
they are or where they are going. This could be problematic for users who, like some of the
evaluators, may confuse the quote feature and the insurance score features.
Consider changing the "Step 1 of 2" label to include "Step 1 of 2 of Insurance Score" or provide a
clear header for the form pages.
H6.1
MILD
22.
NEGATIVE FINDINGS

23. H8: Aesthetic and
minimalist design
X DIALOGUES SHOULD NOT CONTAIN
INFORMATION WHICH IS IRRELEVANT OR
RARELY NEEDED.
H8.1 The copy is wordy on the home page. Irrelevant words can
be edited to reduce the amount of words for the user.
Consider editing the copy for brevity (e.g. "save some money"
could be "save money" or "get the coverage" could be "get
coverage.")
H8.2 The headline "clear-cut car insurance starts with a click" is
not specific or precise, although that is what clear-cut usually
means. The word click is understood by mobile users, but it is
not accurate.
Consider changing the headline to make sense for users on all
platforms.
H8.2
H8.1
MODERATE
23.
NEGATIVE FINDINGS

24. NEGATIVE FINDING RECOMMENDATION RANKING
H3
H3.1 The user who inputs all of the fields accurately is disappointed to reach the end only
to find that they are not able to obtain an Insurance Score because they are out of service
areas. This was the only item ranked as a severe usability problem.
Consider letting users know earlier whether they are in service areas or not before
they provide any personal information or consider a state field in the address
section with a drop-down that only includes the states serviced by Say.
4
H4
H4.1 The main menu changes between the home page and the secondary level pages of
the insurance score. The color changes and the secondary menu disappears. While this
minimizes the chance the user will move out of the Insurance Score, it is inconsistent.
Consider maintaining consistency in the main and secondary menu's colors and
behaviors.
H5
H5.1 Errors are not prevented. Errors show up after the users submit their information.
Required fields are not indicated, and hint text inside the fields are not provided. Consider stopping users from being able to select the "next" button to go on to
the next page until all inputs have been filled in correctly.
H6
H6.1 Once the user starts the Insurance Score form, nothing on the screen tells the user
where they are or where they are going. This could be problematic for users who, like
some of the evaluators, may confuse the quote feature and the insurance score features.
Consider changing the "Step 1 of 2" label to include "Step 1 of 2 of Insurance Score"
or provide a clear header for the form pages.
H8
H8.1 The copy is wordy on the home page. Irrelevant words can be edited to reduce the
amount of words for the user.
Consider editing the copy for brevity (e.g. "save some money" could be "save
money" or "get the coverage" could be "get coverage").
H8
H8.2 The headline "clear-cut car insurance starts with a click" is not specific or precise,
although that is what clear-cut usually means. The word click is understood by mobile
users, but it is not accurate.
Consider changing the headline to make sense for users on all platforms.
RECOMMENDATIONS SUMMARY
SEVERE
MILD
MILD
MILD
MODERATE
MODERATE
24.
RECOMMENDATION SUMMARY

25. THIS WORK WAS PERFORMED AS COURSEWORK IN THE UNIVERSITY OF CALIFORNIA-
IRVINE'S MHCID PROGRAM. SAY IS NOT THE AUTHOR'S CLIENT AND DID NOT HIRE THE
AUTHOR FOR THIS WORK. ALL IDEAS AND OPINIONS ARE THE AUTHOR'S ALONE.
Prepared for Informatics 283: User Experience
by Maria Haynie
mhaynie1@UCI.edu
25.
END