SlideShare a Scribd company logo
1 of 32
1
April 15, 2015
Written by:
Jessica Gray
Juan Hernandez
Taylor Spencer
2
Contents
List of Tables 3
List of Figures 3
Executive Summary 4
Introduction 5
Methodology 6
Sessions 6
Participants 7
Evaluation Tasks 8
Results 9
Task Completion Success Rate 9
Task Difficulty 9
1. Looking up information about Parasitic Diseases 10
2. Family traveling internationally and inquiring necessary vaccines and risks10
3. Vaccines and side effects for babies 11
4. How quickly an outbreak can spread and how to contain it 12
5. Risk of radiation poisoning and safe levels of exposure 13
6. Injury statistics 13
Time on Task 14
Errors 14
Data Summary 15
Recommendations 16
Conclusion 16
Appendix 17
Pre-Test Questionnaire 17
Detailed Usability Script 19
Pre-Test 19
During Testing 19
Post-Test Questionnaire 20
After the Test 20
Tasks 20
Post-Test 21
Raw Usability Test Data 23
Participant #1 25
Post-Test Questionnaire 27
Participant #2 27
Post-Test Questionnaire 29
Participant #3 29
Post-Test Questionnaire 31
3
List of Tables
Table 1: Participant Demographics 7
Table 2: Internet Experience 7
Table 3: Task Completion Rates 9
Table 4: Time to Complete Each Task 13
Table 5: Number of Errors in Each Task 14
Table 6: Participant Pre-Test responses 22
Table 7: Participant Likert Responses 22
Table 8: Participant Comments 23
List of Figures
Figure 1: Task 1 Ratings 10
Figure 2: Task 2 Ratings 10
Figure 3: Task 3 Ratings 11
Figure 4: Task 4 Ratings 11
Figure 5: Task 5 Ratings 12
Figure 6: Task 6 Ratings 12
4
Executive Summary
For our usability test, we tested the website of the Center for Disease Control which is the CDC’s
primary communication channel. It exists to provide credible information to the general public
on diseases and outbreaks as well as information relevant to researchers, public health officials,
and the media. We recruited three participants, and each session lasted about twenty-six minutes.
The demographic of the participants were as follows:
 2 males aged 20-23 and 1 female aged 20-23
 1 English major, 1 Electrical Engineering major, and 1 Art Major
 1 Senior and 2 Juniors
 2 Participants frequented 0-2 hours of computer use and 1 frequented 3-5 hours
 1 Participant had visited the CDC website in the past
All participants were asked to complete six scenarios and only one participant was able to
successfully complete all six scenarios. The other two participants could not complete scenario 5.
Each participant was asked to rate each scenario using very easy/easy, neither, and difficult/very
difficult. The following is the difficulty rating for each scenario:
 Scenario 1: 67% “Easy” and 33% “Neither”
 Scenario 2: 67% “Very Easy” and 33% “Easy”
 Scenario 3: 34% “Very Easy”, 33% “Easy”, and 33% “Neither”
 Scenario 4: 67% “Neither”, 33% “Easy”
 Scenario 5: 33% “Very Difficult”, 33% “Difficult” and 33% “Easy”
 Scenario 6: 100% “Easy”
While going through the scenarios, we asked each participant to voice their opinions as they
went through the site. One common complaint was display of information, especially regarding
charts as some of them did not have a key to let the participant know what each color
represented. Along with that, some of the information was scattered along the page and
participants also expressed their distaste with being redirected to different web pages, sometimes
not the CDC’s, when clicking some of the links instead of the web page scrolling down to where
to information should have been.
All participants eventually had to use the search bar in order to find some of the information in
the scenarios. In some cases, the links presented to participants did not contain the key words
they searched for or links that had the all the keywords were ranked lower than the ones that had
only one or two key words.
Overall, the participants found the scenarios easy but did not like the layout of the site and the
presentation of information. Based on our participant’s complaints, we have several
recommendations for the CDC most of which are regarding the display of information or the
overwhelming amount of information.
5
Introduction
We tested the usability of the CDC’s website which contains information on various diseases and
conditions, data and statistics on said diseases, as well as information on healthy living and
travel. This website is designed for anyone, regardless of background, to access and learn. There
is information for public health professionals, researchers, the general public and anyone in
between.
We conducted our usability test with three participants and used six scenarios. Each member of
our group played a different role in conducting this test. Juan Hernandez facilitated each session,
Jessica Gray took notes, and Taylor Spencer timed each scenario. We all worked together to
compile the following report on our findings.
6
Methodology
Sessions
The facilitator recruited three friends to be participants for the usability test. The facilitator then
stated the purpose of the usability test, asked for their participation, and stated the amount of
time that will be used for testing. During the session, the facilitator explained the test and asked
the participant to fill out a pre-test questionnaire (see appendix). Afterwards, the facilitator read
the tasks, and the participants tried to find each of the tasks on the website while explaining what
they were doing and thinking. Each individual task lasted approximately two to six minutes.
After each task was completed, the facilitator asked each of the three participants to rate the
website on a 5-point Likert Scale with measures ranging from Very Easy to Very Difficult. We
also asked them to explain why they chose a certain rating, which was all included in the post-
test questionnaire (see appendix). After the last task was completed, the facilitator asked the
participants to describe any specific comments or concerns they might have about the website.
7
Participants
All of the participants are friends of the facilitator, Juan Hernandez. The three participants were
tested on April 8, 2015 at around 3:30pm. Table 1 shows the demographics of the three
participants, and Table 2 shows the Internet Experience of the three participants.
Table 1: Participant Demographics
Demographics Participant 1 Participant 2 Participant 3
Gender Male Male Female
Age 20 23 22
Table 2: Internet Experience
Internet Experience Participant 1 Participant 2 Participant 3
Frequency of
Computer Use
(Daily)
0-2 Hours 3-5 Hours 0-2 Hours
Searched Online for
Information About
Symptoms of Illnesses
and Other
Diseases, Symptoms of
Illnesses, Vaccines,
Safety Information, and
Other
Diseases, Symptoms of
Illnesses, Vaccines,
Safety Information, and
Other
Completed science
related courses
Not Applicable Biology and Other Biology and Anatomy
and Physiology
Visitation of certain
medical websites
Other Other Mayoclinic.org and
cdc.gov
8
Evaluation Tasks
For our usability test, we decided to create six tasks that require someone to browse through the
CDC.gov website to find specific information. We image that people who visit the site will come
looking for something specific and will look at the top navigation tabs first. The following is a
list of the tasks the participants completed and why each task was created:
 Looking up information about parasitic diseases
o The CDC website has immeasurable amounts of information about all types of
diseases. We included this task to see how easy it would be to find information
about parasitic diseases including, symptoms, location of origination, and how to
treat it.
 Family traveling internationally and inquiring necessary vaccines and risks
o People want to know how to prepare to travel internationally and how to protect
their health. We included this task because the CDC website provides a specific
area for people to select their destination, along with specificities, to give them
the updated report of what vaccines are necessary and what they’re at risk for.
 Vaccines and side effects for babies
o Children, especially newborns, are highly susceptible to diseases because of their
weak immune system. We included this task because it is essential for mothers to
have current and correct information about vaccines to protect their children.
 How quickly an outbreak can spread and how to contain it
o The United States is a popular place for international tourists to visit, some may
not have the proper vaccinations and bring diseases with them, potentially
affecting natives. We included this task because it is important to know how fast
an outbreak may spread and how to contain it in case an outbreak plagues the
nation.
 Risk of radiation poisoning and safe levels of exposure
o Radiation is found in common areas that most may not be aware of. We included
this task because it is vital for people to know how to treat radiation poisoning,
how to find out about different types of radiation in homes, and how much is too
much exposure.
 Injury statistics
o Injuries happen daily and are sometimes fatal or nonfatal. We included this task
because people should know how to prevent injuries or react when injuries occur.
9
Results
Task Completion Success Rate
Two participants experienced difficulty completing the fifth task. None of our participants spend
a significant amount of time on the internet and are not experienced navigating through a
medical web site. Due to this information, it proves that some tasks were too hard to find and
users became frustrated and gave up or thought it took too much time and effort when they
finally did completed the task.
Table 3: Task completion rates
Participant Task 1 Task 2 Task 3 Task 4 Task 5 Task 6
1 Y Y Y Y N Y
2 Y Y Y Y Y Y
3 Y Y Y Y N Y
# Of Success 3 3 3 3 1 3
Completion
Rate
100% 100% 100% 100% 33.33% 100%
Task Difficulty
The following procedure was performed to test each task with each participant individually. The
participants listened to the task read aloud by the facilitator, performed the task, and were asked
to fill out the corresponding question in the post-test questionnaire. In the questionnaire,
participants rated the level of difficult on a Likert scale and added comments.
10
1. Looking up information about parasitic diseases
Each participant chose different ways to look up information about parasitic diseases. Participant
#1 and #3 correctly clicked on the Diseases and Conditions tab, which easily led them to find
information about different parasitic diseases. Participant #2 chose to use the search bar; the
search results did not provide the best or most accurate answer, though still had useful
information. Participant #3 commented that using the search bar did not “give me desired
results.” Participant #2 whom used the search bar thought it “made it easy to find information.”
The Likert results for this task are seen in the following pie chart.
2. Family traveling internationally and inquiring necessary vaccines and risks.
All participants correctly selected Traveler’s Health and selected their destination, specificities,
and received accurate information about vaccines and risks for international travel. 1 participant
felt the selection boxes for the specifics made the search much easier. The Likert results for this
task are seen in the following pie chart.
67%
33%
0%0%0%
Task 2
Very Easy
Easy
Neither
Difficult
Very Difficult
0%
67%
33%
0%0%
Task 1
Very Easy
Easy
Neither
Difficult
Very Difficult
Figure 1: Task 1 Ratings
Figure 2: Task 2 Ratings
11
3. Vaccines and side effects for babies
As participants familiarized themselves with the site, they recognized which tabs would have
what type of information. Participants #1 and #3 clicked on Healthy Living and selected
Vaccines and Immunizations, which is what our team selected when performing the tasks prior
to the test. It was somewhat confusing and difficult for them to find accurate information, but
eventually both found some useful information. Surprisingly, Participant #2 found the chart our
team found by using the search bar. The other two participants did not find that chart, which had
the most detailed and prominent information. The Likert results for this task are seen in the
following pie chart.
34%
33%
33%
0%0%
Task 3
Very Easy
Easy
Neither
Difficult
Very Difficult
Figure 3: Task 3 Ratings
12
4. How quickly an outbreak can spread and how to contain it
Participants #2 and #3 did not click on the correct tabs and therefore, did not receive the best
information about outbreaks. One participant had to scroll through news articles that did not
relate to outbreaks in the U.S., only significant international plagues. It was unhelpful for their
search. The other participant used the search bar and only received information about one
specific outbreak. Overall, two of the three participants did not know how they felt about this
task. The Likert results for this task are seen in the following pie chart.
0%
33%
67%
0%0%
Task 4
Very Easy
Easy
Neither
Difficult
Very Difficult
Figure 4: Task 4 Ratings
13
5. Risk of radiation poisoning and safe levels of exposure
Participant #1 correctly clicked on the Emergency Preparedness tab. One participant clicked on
the Healthy Living tab while the other used the search bar. Overall, none of the participants
found the information that our team found, which provided accurate and a sufficient amount of
information. Two out of the three participants found this task to be difficult or very difficult. On
average, this task took the longest. The Likert results for this task are seen in the following pie
chart.
6. Injury statistics
Every participant correctly clicked on Diseases and Conditions, which led to a Data/Statistics
tab. From there, each participant chose different statistical charts to look at, which was accepted
because it was not specified. Each participant thought this task was easy to navigate and thought
it was very straight forward. The Likert results for this task are seen in the following pie chart.
0%
100%
0%0%0%
Task 6
Very Easy
Easy
Neither
Difficult
Very Difficult
0%
34%
0%
33%
33%
Task 5
Very Easy
Easy
Neither
Difficult
Very Difficult
Figure 5: Task 5 Ratings
Figure 6: Task 6 Ratings
14
Time on Task
The longest time a participant took on a task was 6 minutes and 25 seconds, which was Task 4.
While participant #1 took 6 minutes and 25 seconds, the other two participants were able to
complete between 4 to 5 minutes. Similarly, participant #1 took 4 minutes and 21 seconds to
complete Task 6 while the other two participants took about half of that time to complete it. It
appeared that each participants took more or less time on different tasks than the other
participants, so none of the tasks took around the same time for all three participants. Another
example of this can be seen in Task 1 in which one participant completed the task in less than
two minutes while the other participants took between 2 to 4 minutes longer.
Table 4: Time taken to complete each task
Participant Task 1 Task 2 Task 3 Task 4 Task 5 Task 6
1 235s 370s 205s 385s 353s 261s
2 310s 375s 247s 330s 221s 141s
3 110s 147s 243s 247s 368s 192s
Total Average 218s 297s 232s 321s 314s 198s
Errors
Our group initially completed the tasks individually and wrote down exactly what we clicked.
Based upon that information, each participant had a large amount of errors in almost every task.
One participant relied solely on using the search bar for every task. Overall, in four out of the six
tasks, 2/3 of participants made errors. In every task, there was at least 1 participant who made no
errors. Other than Tasks 3 through 6, nobody made more than 1 error. Task #2 was completed
without any errors by all three participants.
Most errors resulted from clicking the wrong navigation bar tab.
The amount of information on the different pages caused several errors. Some titles of articles
were too long and specific that participants did not want to read them. Participants had to search
closely to find enough information to complete the task. N/A means the participant was unable to
complete the task.
15
Table 5: Number of errors in each task
Participant Task 1 Task 2 Task 3 Task 4 Task 5 Task 6
1 0 0 0 0 N/A 1
2 3 0 1 3 3 3
3 0 0 6 4 N/A 0
Data Summary
Each participant selected “Easy” for three of the six tasks completed, therefore the usability test
was easy overall for our participants. Each participants had difficult in different areas since they
all used different methods to complete the specified task.
As seen in Figure 1 in the Task Difficulty section above, Task 1 (Looking up information about
Parasitic Diseases) had a 67% “easy” rating, and a 33.33% “Neither” rating.
Task 2 had a 67% “Very Easy” rating compared to the 33.33% “Easy” rating, the participant that
put “Easy” said the “topics were well outlined.”
Task 3 had a variety of difficulty ratings. 34% (1 participant) selected “Very Easy”, 33% (1
participant) selected “Easy”, and 33% (1 participant) selected “Neither.”
Task 4 had a 67% “Neither” rating and a 33% “Easy” rating. It would be accurate to conclude
that many ratings have a two to one ratio of agreeance.
Task 5 had another three-way split with 33% “Very Difficult”, 33% “Difficult” and 33% “Easy.”
This task provided the most difficulty with our participants. Two of the three participants
resulted with inconclusive evidence because they exhausted all their options and in the end, did
not find the information they were looking for.
There was a 100% success rate for Task 6. All participants noted that it was easy, straight
forward, and it was easy to find.
16
Recommendations
Based on the results of our usability test and the feedback we received from the participants we
have numerous recommendations for the CDC.gov website. Our first suggestion regards the
links in the navigation bar that can be found throughout the website. One of our test participants
repeatedly found them difficult to click on.
This leads us to the Diseases and Conditions page. Our participants found the list to be a little
too comprehensive, and the video on the page did not help, as the participants didn’t like both
that it played automatically and that it didn’t have any sound. A recourse to this might be to
allow the user to show and hide sections of the page as they need it. As for the video, we would
suggest getting rid of it and replacing it with an animated GIF, as there’s no point of a video
without sound. Also, the Symptoms and Treatments pages should be merged, with the resultant
page becoming the first page. Lastly, one participant wondered why they could only find scabies
in the Parasites list. Participants, however, were able to find the information they were looking
for.
For the Traveler’s Health page, the drop-down list for this is too long. Considering the length of
destinations, a drop-down list probably shouldn’t be used for this. Also, our testers wanted more
information on the Traveler’s Health page.
From our third scenario, we came to the conclusion that there should be a section for new
mothers, as our participants had difficulty finding information for this. When our participants
did find the information, they noticed that the “Before, During, and After” section only had
information on what vaccines were needed before a pregnancy. We recommend that information
on what vaccines would be necessary after a pregnancy should be added, along with information
on which vaccines would be needed by infants.
Our participants were able to quickly find information on an outbreak, however the presentation
of this information could use some work.
Conclusion
For most of our tasks, our participants navigated the CDC.gov website easily. However, they
were displeased with the presentation of this information. Among their complaints, they thought
the navigation links needed to be easier to click on, they found information scattered in different
places, and they didn’t find information on the pages they expected to find them on. In one case,
they couldn’t find information on the healthy level of radiation. However, in most cases, the
information was there, it was just poorly presented and scattered all over the place. With this in
mind, we think that the best course of action for the CDC is reorganization and redesign the
website so that it places the information people need where people would expect to find it.
17
Appendix
Pre-Test Questionnaire
Name (please print): ___________________________________________
1. What is your gender?
❏ Male
❏ Female
2. What is your age?
________
3. How many hours on average do you spend on the Internet each day?
❏ 0-2 hours
❏ 3-5 hours
❏ 6-8 hours
❏ Over 9 hours
4. Do you go online to find information about any of the following provided (check all that
apply):
❏ Diseases
❏ Symptoms of illnesses
❏ Vaccines
❏ Safety Information
❏ Other
❏ Not Applicable
5. Which of the following science related courses have you completed? (check all that apply)
❏ Biology
❏ Anatomy & Physiology
❏ Microbiology
❏ Other
❏ Not Applicable
6. Have you visited any of the following websites? (check all that apply):
❏ mayoclinic.org
❏ cdc.gov
❏ niaid.nih.gov
❏ familydoctor.org
❏ fema.gov
❏ Other
18
❏ Not Applicable
19
Detailed Usability Script
Hello and thank you for volunteering to help us out. My name is Juan Hernandez and I am here
today with two colleagues, Jessica and Taylor. They will be taking notes as I guide you through
this session.
We are asking people to evaluate the user-friendliness of the Center for Disease Control’s
website. I would like to take a moment to explain what you should expect. Before the evaluation,
I will read you a scenario and you will navigate the site accordingly. There are six scenarios, but
the evaluation should not take longer than thirty minutes. We are not evaluating your skills as the
user, so do not worry about making mistakes. We are only evaluating how you navigate the site,
and what your thoughts are about it. Your feedback today will help us evaluate this website.
During the evaluation, I ask that you please think out loud. Hearing the way you perceive what
you are seeing is important to our evaluation of the site. What we ask is that you say what you
are looking at, what you are trying to do, and what you are thinking. I understand that thinking
out loud seems strange, but it is important that we hear why you make the choices that you do,
and what thoughts you have while you are navigating the site. Don’t worry about offending
anyone in the room, we want to hear your honest thoughts.
Since we are evaluating the site, not you, feel free to ask any questions out loud, but I may not
answer them right away. We are trying to simulate normal circumstances in which people use
this site. If you need to take a break at any point, please let me know. If we reach the end of the
evaluation and you still have questions, I’ll try to answer them. Afterwards, you will complete a
quick post-test survey and this session will be complete.
Thank you again for your time. Do you have any questions before we begin?
Let’s get started!
Pre-test Questionnaire
Before we begin, we would appreciate it if you took this pre-test questionnaire so that we will be
able to see how much Internet and medical website experience you have. This will help evaluate
the results by showing up your amount of experience. Thank you for filling out the
questionnaire.
During Testing
It is important that I must remind all of you to think out loud and say why you are choosing to
navigate the site as you are. I will present different tasks for all of you to solve using the site.
When the tasks are completed we will pass out the post-test questionnaire, which is similar to the
pre-test questionnaire that will help evaluate the results.
20
Post-test Questionnaire
Thanks for all of your help; we are almost done with the evaluation. Here is the post-test
questionnaire that I mentioned earlier. Please fill out the questionnaire so that we can evaluate
today’s results. You will be marking the difficulty of each task, along with a brief explanation of
why.
After the Test
Thank you for your help today. This information will be very helpful as we evaluate all the
results that have been gathered, have a great day!
Tasks
1. In your biology class, you heard about parasitic diseases, and you wanted to research it
further.
2. Your neighbor, his wife, and two children are planning a trip to Europe and want to know
what vaccines are needed and what diseases they’re at risk of.
3. Your friend Jane recently had a baby and wants to know the necessary vaccines and
potential side effects.
4. You’re in your statistics class and start discussing a recent outbreak plaguing the nation.
You want to find more information about how quickly an outbreak spreads and how
quickly it can be contained.
5. You want to work as a radiation engineer and need to find information about the risk of
radiation poisoning and a safe level of exposure.
6. You’re writing a paper for class about injury statistics, and want to find more
information.
21
Post-Test Questionnaire
Name (please print): ___________________________________________
1. How easy or difficult was it to find information about a disease?
Very Easy Easy Neither Difficult Very difficult
Why was it easy or difficult?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
2. How easy or difficult was it to find information about vaccines and risks?
Very Easy Easy Neither Difficult Very difficult
Why was it easy or difficult?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
3. How easy or difficult was it to find information about vaccines and side effects for
babies?
Very Easy Easy Neither Difficult Very difficult
Why was it easy or difficult?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
22
4. How easy or difficult was it to find about an outbreak?
Very Easy Easy Neither Difficult Very difficult
Why was it easy or difficult?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
5. How easy or difficult was it to find information about radiation?
Very Easy Easy Neither Difficult Very difficult
Why was it easy or difficult?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
6. How easy or difficult was it to find information about injury statistics?
Very Easy Easy Neither Difficult Very difficult
Why was it easy or difficult?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
23
Raw Usability Test
Table 6: Participant pre-test responses
Participant 1 Participant 2 Participant 3
Q1 Male Male Female
Q2 20 23 22
Q3 0-2 hours 3-5 hours 0-2 hours
Q4 Symptoms of Illnesses
and Other
Diseases, Symptoms of
Illnesses, Vaccines, Safety
Information, and Other
Diseases, Symptoms of
Illnesses, Vaccines, Safety
Information, and Other
Q5 Not Applicable Biology and Other Biology and Anatomy and
Physiology
Q6 Other Other Mayoclinic.org and CDC.gov
Table 7: Participant Likert Responses
Very Easy Easy Neither Difficult Very Difficult
Q1 0 2 1 0 0
Q2 2 1 0 0 0
Q3 1 1 1 0 0
Q4 0 1 2 0 0
Q5 0 1 0 1 1
Q6 0 3 0 0 0
24
Table 8: Participant Comments
Participant 1 Participant 2 Participant 3
Q1 “It has basic information so
it could work alright.”
“I don’t care where the sidebar
links are but its location is
unique.”
“Why is Scabies the only kind of
parasite listed?”
Q2 “I mean yeah, most of this
information seems pretty
basic.”
“I would like if they had a
suggestion area a ‘help us help
you’ recommendation section.”
“I’m going to select immune
compromised in case there are
children or older people.”
Q3 “The page should be bigger
and more condensed.”
“The hyperlink needs a
different font because I
wouldn’t know it was a
hyperlink.”
“It has information for before and
during pregnancy but not after.”
Q4 “I like the pamphlet it’s
colorful, catches the eye,
and easy to read.”
“I wish they had links with the
key words listed first.”
“Oh this is talking about
foodborne illnesses, this wasn’t
what I was expecting.”
Q5 “I have no idea how to
answer this, I even typed in
the search bar and still
nothing.”
“The picture gives off decent
amount of information.”
“This opened up a different
website.”
Q6 “There is no legend for the
colors, I have no idea what
this means.”
“I think the chart, it says a lot.” “I don’t wanna fill out a form to
get information.”
25
Participant #1
Pre-test
1. Male
2. 20
3. 0-2 hours
4. Symptoms of Illnesses, and Other
5. Not Applicable
6. Other
Task #1
● Clicked on Diseases and Conditions
● Clicked More
● Searched “J” for Japanese Encephalitis
● Clicked Symptoms
● Time: 3 minutes and 55 seconds
● Mistakes: None
Task #2
● Clicked on Traveler’s Health
● Clicked on Destinations
● Selected United Kingdom
● Selected Traveling with Kids
● Selected Pregnant
● Pressed Go
● Time: 6 minutes and 10 seconds
● Mistakes: None
Task #3
● Clicked Healthy Living
● Clicked Vaccines and Immunizations
● Typed “pregnant woman” in the search bar
● Selected Vaccine Information for Adults link
● Selected List of Vaccines
● Time: 3 minutes and 25 seconds
● Mistakes: None
Task #4
● Clicked Emergency Preparedness
● Clicked Recent Outbreaks
● Selected Illness
● Selected Infection Control
● Selected Ebola Outbreak
● Time: 6 minutes and 25 seconds
● Mistakes: None
26
Task #5
● Clicked Emergency Preparedness
● Clicked Radiation Emergency
● Clicked Radiation Health
● Clicked Questions about Radiation
● Clicked Impact of Radiation in Daily Lives
● Clicked Learn more
● Clicked Information about Professionals
● Starts from the homepage
● Clicked Healthy Living
● Clicked More
● Typed “safe radiation exposure” in search bar
● Selected Radiation Emergencies link
● Clicked More Information
● Clicked Environmental Health
● Clicked Investigations
● Clicked General Radiation
● Time: 5 minutes and 53 seconds
● Mistakes: N/A
Task #6
● Clicked Healthy Living
● Clicked back to the homepage
● Clicked Diseases and Conditions
● Clicked Data/Statistics
● Clicked Injuries/Violence
● Selected Non-fatal Data
● Clicked on pictures
● Selected Leading Causes of Death by Age Groups
● Time: 4 minutes and 21 seconds
● Mistakes: 1
o Clicked on Healthy Living, realized it was wrong, and clicked on Diseases and
Conditions
27
Post-test
1. Easy
○ Alphabetic list.
2. Very Easy
○ Quick search with selection boxes.
3. Easy
○ Searched for pregnant women under vaccines.
4. Neither
○ Obscure directory.
5. Very Difficult
○ Could not find the information I needed.
6. Easy
○ Injury button from homepage.
Participant #2
Pre-test
1. Male
2. 23
3. 3-5 hours
4. Diseases, Symptoms of Illnesses, Vaccines, Safety information, and Other
5. Biology, and Other
6. Other
Task #1
● Typed “parasitic diseases” in search bar
● Clicked on the first link
● Selected “L”
● Selected CDC in Lebanon
● Clicked Travel
● Time: 5 minutes and 10 seconds
● Mistakes: 3
○ Did not click on Diseases and Conditions, did not select More, and did not find
information about parasitic diseases.
Task #2
● Clicked Traveler’s Health
● Clicked Destinations
● Selected Ireland
● Clicked Go
● Time: 6 minutes and 15 seconds
● Mistakes: 0
28
Task #3
● Typed “newborn vaccine” in the search bar
● Selected For Everyone
● Selected Infants and Children
● Clicked 2015 Immunizations Recommended for Children Newborn to 6 Years Old
● Time: 4 minutes and 7 seconds
● Mistakes: 1
○ Did not click on Healthy Living
Task #4
● Clicked Diseases and Conditions
● Typed “plague outbreaks” in the search bar
● Selected CDC Current Outbreak List
● Looked at the Breakdown of U.S. and International
● Selected Cyclospinosis-Parasites
● Time: 5 minutes and 30 sec
● Mistakes: 3
○ Did not click on Emergency Preparedness, did not select Recent Outbreaks and
Incidents, did not use the A-Z Index
Task #5
● Typed “radiation levels” in the search bar
● Selected Exposure/Levels
● Clicked U.S. Environmental Protection Agency link
● Clicked on the link under the picture
● Found a Relative Doses Chart
● Time: 3 minutes and 41 sec
● Mistakes: 3
○ Did not click on Emergency Preparedness, did not select Radiation emergencies,
and did not click on Radiation and Your Health
Task #6
● Clicked Emergency Preparedness
● Selected More CDC Topics
● Clicked Injury, Violence, and Safety
● Time: 2 minutes and 21 sec
● Mistakes: 3
○ Did not click on Diseases and Conditions, did not select Data and Statistics, and
did not select Injury Prevention and Control
29
Post-test
1. Easy
○ Search bar made it easy to find but menu options did not seem directly related.
2. Very Easy
○ Directly from home page, the info is easy to search.
3. Very Easy
○ Straight forward, search & info
4. Neither
○ Would prefer more visual for quick search information
5. Easy
○ Unexpected finding but info was easy to understand.
6. Easy
○ Straight forward in terms of searching
Participant #3
Pre-test
1. Female
2. 22
3. 0-2 hours
4. Diseases, Symptoms of Illnesses, Vaccines, Safety Information, and Other
5. Biology and Anatomy and Physiology
6. Mayoclinic.org and CDC.gov
Task #1
● Clicked Diseases and Conditions
● Selected More
● Clicked Parasites-Scabies
● Clicked on related links
● Selected the A-Z Index for parasites in general
● Typed “about parasites” in the search bar
● Time: 1 minute and 50 seconds
● Mistakes: None
Task #2
● Clicked Traveler’s Health
● Clicked Destinations
● Clicked Travelers
● Selected England
● Selected Traveling with kids
● Selected Pregnant
● Selected Immune Compromised
30
● Read Routine Vaccines
● Time: 2 minutes and 27 seconds
● Mistakes: None
Task #3
● Clicked Conditions and Diseases
● Went back to the homepage
● Clicked Healthy Living
● Clicked Vaccines/Immunizations
● Clicked Vaccine Side Effects and Safety
● Selected Pregnant Women
● Selected Infants and Toddlers
● Selected Vaccines for Pregnant Women
● Selected Vaccines Before, During, and After Pregnancy
● Clicked Adult Vaccines
● Clicked on “I” for Infants
● Time: 4 minutes and 3 seconds
● Mistakes: 6
○ First clicked Conditions and Diseases, selected Pregnant Women, selected
Vaccines for Pregnant Women, selected Vaccines Before, During, and After
Pregnancy, selected Adult Vaccines, selected “I” for Infants.
Task #4
● Clicked Diseases and Conditions
● Selected More
● Selected More CDC Topics
● Goes back to the homepage
● Selected Emergency Preparedness
● Selected Recent Outbreak News Article
● Selected 2006 Outbreak
● Selected 2014 West Africa Ebola Article
● Typed “what to do in an outbreak” in the search bar
● Clicked on Foodborne Illnesses
● Typed “outbreak preparedness” in the search bar
● Clicked Water-Related Emergencies
● Clicked on Outbreak Response
● Time: 4 minutes and 7 seconds
● Mistakes: 4
○ Did not click Emergency Preparedness, did not select Recent Outbreaks and
Incidents, did not use the A-Z Index, and did not find information about
responses to plagues.
31
Task #5
● Clicked on Healthy Living
● Clicked on more
● Clicked on Specific populations
● Clicked on Workplace Safety Information
● Selected “R” for Radiation
● Selected Radiation Dose Reconstruction
● Typed “safe radiation levels” in the search bar
● Clicked back to the homepage
● Clicked Healthy Living
● Clicked More
● Clicked Workplace Safety Information
● Clicked Hazards/Exposure
● Clicked UV Radiation
● Clicked back to Hazards/Exposure
● Clicked back to Workplace Safety Information
● Clicked Safety and prevention
● Clicked Industries and Occupations
● Typed “safe nuclear radiation levels” in the search bar
● Clicked the first link about Japan
● Linked to OSHA.com
● Time: 6 minutes and 8 seconds
● Mistakes: Not Applicable
Task #6
● Clicked Diseases and Conditions
● Clicked Data/Statistics
● Clicked Injury/Violence
● Selected Fatal Injury Data
● Selected Injury Report 1999-2013
● Clicked Leading Causes
● Filtered: Female, Years 2000-2000, Treated and Released
● Submitted Request
● Time: 3 minutes and 12 seconds
● Mistakes: None
32
Post-test
1. Neither
○ There was too much information to personally sort through. Using the search bar
did not give me desired results- it gave me articles instead of general date.
2. Easy
○ Topics were well outlined
3. Neither
○ Information on babies was not closely correlated with the information on pregnant
women
4. Easy
○ Menu options allowed easy access
5. Difficult
○ Website did not list nuclear radiation specifically- I ended up at a completely
different website.
6. Easy
○ Menu options made it easy, but the data was very messy.

More Related Content

Similar to Usability Test

Written College Essays. Online assignment writing service.
Written College Essays. Online assignment writing service.Written College Essays. Online assignment writing service.
Written College Essays. Online assignment writing service.Lori Gilbert
 
IMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATION
IMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATIONIMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATION
IMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATIONArif, Mohammed Nazrul Islam
 
Utilization Based Evaluation In Mental Health
Utilization Based Evaluation In Mental HealthUtilization Based Evaluation In Mental Health
Utilization Based Evaluation In Mental HealthPaul Frankel
 
Using Social Media to Support Youth Healthy Sexual Behavior
Using Social Media to Support Youth Healthy Sexual BehaviorUsing Social Media to Support Youth Healthy Sexual Behavior
Using Social Media to Support Youth Healthy Sexual BehaviorYTH
 
Proposal-Example-4.doc
Proposal-Example-4.docProposal-Example-4.doc
Proposal-Example-4.docmebrahten1
 
Proposal-Example-4.doc
Proposal-Example-4.docProposal-Example-4.doc
Proposal-Example-4.docAyesha Lata
 
Figure 2. Sample Gantt ChartActivitiesDatesNovember 1,20.docx
Figure 2.   Sample Gantt ChartActivitiesDatesNovember 1,20.docxFigure 2.   Sample Gantt ChartActivitiesDatesNovember 1,20.docx
Figure 2. Sample Gantt ChartActivitiesDatesNovember 1,20.docxlmelaine
 
PERI-Facts.11.2.11
PERI-Facts.11.2.11PERI-Facts.11.2.11
PERI-Facts.11.2.11Bob Latino
 
Examples Of University Application Essay
Examples Of University Application EssayExamples Of University Application Essay
Examples Of University Application EssaySue Jones
 
Business research report on Internet and children
Business research report on Internet and childrenBusiness research report on Internet and children
Business research report on Internet and childrenApon Comilla
 
IRJET- An Empirical Study of Social Media Addiction
IRJET- An Empirical Study of Social Media AddictionIRJET- An Empirical Study of Social Media Addiction
IRJET- An Empirical Study of Social Media AddictionIRJET Journal
 
The Anotated bilography should look like this Social1) S.docx
The Anotated bilography should look like this Social1) S.docxThe Anotated bilography should look like this Social1) S.docx
The Anotated bilography should look like this Social1) S.docxmattinsonjanel
 
Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27
Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27
Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27Yuanjing Sun
 
Toolkit of recommended curricula and assessments for home visiting
Toolkit of recommended curricula and assessments for home visitingToolkit of recommended curricula and assessments for home visiting
Toolkit of recommended curricula and assessments for home visitingUNICEF Europe & Central Asia
 
Vision Portal Survey Analysis Brief
Vision Portal Survey Analysis BriefVision Portal Survey Analysis Brief
Vision Portal Survey Analysis BriefDiana Woolis
 
Stem V Conference Poster
Stem V Conference PosterStem V Conference Poster
Stem V Conference PosterAnudeeta Gautam
 
Maths survey
Maths surveyMaths survey
Maths surveyxennechau
 
Maths survey (assignment 2)
Maths survey (assignment 2)Maths survey (assignment 2)
Maths survey (assignment 2)Shze Hwa Lee
 

Similar to Usability Test (20)

Written College Essays. Online assignment writing service.
Written College Essays. Online assignment writing service.Written College Essays. Online assignment writing service.
Written College Essays. Online assignment writing service.
 
IMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATION
IMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATIONIMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATION
IMPACT OF SOCIAL NETWORKING SITES ON YOUNG GENERATION
 
Utilization Based Evaluation In Mental Health
Utilization Based Evaluation In Mental HealthUtilization Based Evaluation In Mental Health
Utilization Based Evaluation In Mental Health
 
Using Social Media to Support Youth Healthy Sexual Behavior
Using Social Media to Support Youth Healthy Sexual BehaviorUsing Social Media to Support Youth Healthy Sexual Behavior
Using Social Media to Support Youth Healthy Sexual Behavior
 
Proposal-Example-4.doc
Proposal-Example-4.docProposal-Example-4.doc
Proposal-Example-4.doc
 
Proposal-Example-4.doc
Proposal-Example-4.docProposal-Example-4.doc
Proposal-Example-4.doc
 
Figure 2. Sample Gantt ChartActivitiesDatesNovember 1,20.docx
Figure 2.   Sample Gantt ChartActivitiesDatesNovember 1,20.docxFigure 2.   Sample Gantt ChartActivitiesDatesNovember 1,20.docx
Figure 2. Sample Gantt ChartActivitiesDatesNovember 1,20.docx
 
PERI-Facts.11.2.11
PERI-Facts.11.2.11PERI-Facts.11.2.11
PERI-Facts.11.2.11
 
Examples Of University Application Essay
Examples Of University Application EssayExamples Of University Application Essay
Examples Of University Application Essay
 
Business research report on Internet and children
Business research report on Internet and childrenBusiness research report on Internet and children
Business research report on Internet and children
 
Research Report
Research ReportResearch Report
Research Report
 
IRJET- An Empirical Study of Social Media Addiction
IRJET- An Empirical Study of Social Media AddictionIRJET- An Empirical Study of Social Media Addiction
IRJET- An Empirical Study of Social Media Addiction
 
The Anotated bilography should look like this Social1) S.docx
The Anotated bilography should look like this Social1) S.docxThe Anotated bilography should look like this Social1) S.docx
The Anotated bilography should look like this Social1) S.docx
 
Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27
Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27
Grad1-YuanjingSun-CS5760Evaluation-UtestReport-Apr27
 
Toolkit of recommended curricula and assessments for home visiting
Toolkit of recommended curricula and assessments for home visitingToolkit of recommended curricula and assessments for home visiting
Toolkit of recommended curricula and assessments for home visiting
 
Vision Portal Survey Analysis Brief
Vision Portal Survey Analysis BriefVision Portal Survey Analysis Brief
Vision Portal Survey Analysis Brief
 
Stem V Conference Poster
Stem V Conference PosterStem V Conference Poster
Stem V Conference Poster
 
Maths survey
Maths surveyMaths survey
Maths survey
 
Maths survey
Maths surveyMaths survey
Maths survey
 
Maths survey (assignment 2)
Maths survey (assignment 2)Maths survey (assignment 2)
Maths survey (assignment 2)
 

More from Jessica Fisher

More from Jessica Fisher (12)

rocafinal1-interactive
rocafinal1-interactiverocafinal1-interactive
rocafinal1-interactive
 
Usability Report - Roca Inc.
Usability Report - Roca Inc.Usability Report - Roca Inc.
Usability Report - Roca Inc.
 
Newest-indesign
Newest-indesignNewest-indesign
Newest-indesign
 
8. InDesign Manual-original
8. InDesign Manual-original8. InDesign Manual-original
8. InDesign Manual-original
 
User Experience Poster
User Experience PosterUser Experience Poster
User Experience Poster
 
Submitted-Proposal
Submitted-ProposalSubmitted-Proposal
Submitted-Proposal
 
Poster
PosterPoster
Poster
 
Flyer
FlyerFlyer
Flyer
 
Business Card
Business CardBusiness Card
Business Card
 
Original Flyer
Original FlyerOriginal Flyer
Original Flyer
 
Revised Employee Handbook
Revised Employee HandbookRevised Employee Handbook
Revised Employee Handbook
 
Original Employee Handbook
Original Employee HandbookOriginal Employee Handbook
Original Employee Handbook
 

Usability Test

  • 1. 1 April 15, 2015 Written by: Jessica Gray Juan Hernandez Taylor Spencer
  • 2. 2 Contents List of Tables 3 List of Figures 3 Executive Summary 4 Introduction 5 Methodology 6 Sessions 6 Participants 7 Evaluation Tasks 8 Results 9 Task Completion Success Rate 9 Task Difficulty 9 1. Looking up information about Parasitic Diseases 10 2. Family traveling internationally and inquiring necessary vaccines and risks10 3. Vaccines and side effects for babies 11 4. How quickly an outbreak can spread and how to contain it 12 5. Risk of radiation poisoning and safe levels of exposure 13 6. Injury statistics 13 Time on Task 14 Errors 14 Data Summary 15 Recommendations 16 Conclusion 16 Appendix 17 Pre-Test Questionnaire 17 Detailed Usability Script 19 Pre-Test 19 During Testing 19 Post-Test Questionnaire 20 After the Test 20 Tasks 20 Post-Test 21 Raw Usability Test Data 23 Participant #1 25 Post-Test Questionnaire 27 Participant #2 27 Post-Test Questionnaire 29 Participant #3 29 Post-Test Questionnaire 31
  • 3. 3 List of Tables Table 1: Participant Demographics 7 Table 2: Internet Experience 7 Table 3: Task Completion Rates 9 Table 4: Time to Complete Each Task 13 Table 5: Number of Errors in Each Task 14 Table 6: Participant Pre-Test responses 22 Table 7: Participant Likert Responses 22 Table 8: Participant Comments 23 List of Figures Figure 1: Task 1 Ratings 10 Figure 2: Task 2 Ratings 10 Figure 3: Task 3 Ratings 11 Figure 4: Task 4 Ratings 11 Figure 5: Task 5 Ratings 12 Figure 6: Task 6 Ratings 12
  • 4. 4 Executive Summary For our usability test, we tested the website of the Center for Disease Control which is the CDC’s primary communication channel. It exists to provide credible information to the general public on diseases and outbreaks as well as information relevant to researchers, public health officials, and the media. We recruited three participants, and each session lasted about twenty-six minutes. The demographic of the participants were as follows:  2 males aged 20-23 and 1 female aged 20-23  1 English major, 1 Electrical Engineering major, and 1 Art Major  1 Senior and 2 Juniors  2 Participants frequented 0-2 hours of computer use and 1 frequented 3-5 hours  1 Participant had visited the CDC website in the past All participants were asked to complete six scenarios and only one participant was able to successfully complete all six scenarios. The other two participants could not complete scenario 5. Each participant was asked to rate each scenario using very easy/easy, neither, and difficult/very difficult. The following is the difficulty rating for each scenario:  Scenario 1: 67% “Easy” and 33% “Neither”  Scenario 2: 67% “Very Easy” and 33% “Easy”  Scenario 3: 34% “Very Easy”, 33% “Easy”, and 33% “Neither”  Scenario 4: 67% “Neither”, 33% “Easy”  Scenario 5: 33% “Very Difficult”, 33% “Difficult” and 33% “Easy”  Scenario 6: 100% “Easy” While going through the scenarios, we asked each participant to voice their opinions as they went through the site. One common complaint was display of information, especially regarding charts as some of them did not have a key to let the participant know what each color represented. Along with that, some of the information was scattered along the page and participants also expressed their distaste with being redirected to different web pages, sometimes not the CDC’s, when clicking some of the links instead of the web page scrolling down to where to information should have been. All participants eventually had to use the search bar in order to find some of the information in the scenarios. In some cases, the links presented to participants did not contain the key words they searched for or links that had the all the keywords were ranked lower than the ones that had only one or two key words. Overall, the participants found the scenarios easy but did not like the layout of the site and the presentation of information. Based on our participant’s complaints, we have several recommendations for the CDC most of which are regarding the display of information or the overwhelming amount of information.
  • 5. 5 Introduction We tested the usability of the CDC’s website which contains information on various diseases and conditions, data and statistics on said diseases, as well as information on healthy living and travel. This website is designed for anyone, regardless of background, to access and learn. There is information for public health professionals, researchers, the general public and anyone in between. We conducted our usability test with three participants and used six scenarios. Each member of our group played a different role in conducting this test. Juan Hernandez facilitated each session, Jessica Gray took notes, and Taylor Spencer timed each scenario. We all worked together to compile the following report on our findings.
  • 6. 6 Methodology Sessions The facilitator recruited three friends to be participants for the usability test. The facilitator then stated the purpose of the usability test, asked for their participation, and stated the amount of time that will be used for testing. During the session, the facilitator explained the test and asked the participant to fill out a pre-test questionnaire (see appendix). Afterwards, the facilitator read the tasks, and the participants tried to find each of the tasks on the website while explaining what they were doing and thinking. Each individual task lasted approximately two to six minutes. After each task was completed, the facilitator asked each of the three participants to rate the website on a 5-point Likert Scale with measures ranging from Very Easy to Very Difficult. We also asked them to explain why they chose a certain rating, which was all included in the post- test questionnaire (see appendix). After the last task was completed, the facilitator asked the participants to describe any specific comments or concerns they might have about the website.
  • 7. 7 Participants All of the participants are friends of the facilitator, Juan Hernandez. The three participants were tested on April 8, 2015 at around 3:30pm. Table 1 shows the demographics of the three participants, and Table 2 shows the Internet Experience of the three participants. Table 1: Participant Demographics Demographics Participant 1 Participant 2 Participant 3 Gender Male Male Female Age 20 23 22 Table 2: Internet Experience Internet Experience Participant 1 Participant 2 Participant 3 Frequency of Computer Use (Daily) 0-2 Hours 3-5 Hours 0-2 Hours Searched Online for Information About Symptoms of Illnesses and Other Diseases, Symptoms of Illnesses, Vaccines, Safety Information, and Other Diseases, Symptoms of Illnesses, Vaccines, Safety Information, and Other Completed science related courses Not Applicable Biology and Other Biology and Anatomy and Physiology Visitation of certain medical websites Other Other Mayoclinic.org and cdc.gov
  • 8. 8 Evaluation Tasks For our usability test, we decided to create six tasks that require someone to browse through the CDC.gov website to find specific information. We image that people who visit the site will come looking for something specific and will look at the top navigation tabs first. The following is a list of the tasks the participants completed and why each task was created:  Looking up information about parasitic diseases o The CDC website has immeasurable amounts of information about all types of diseases. We included this task to see how easy it would be to find information about parasitic diseases including, symptoms, location of origination, and how to treat it.  Family traveling internationally and inquiring necessary vaccines and risks o People want to know how to prepare to travel internationally and how to protect their health. We included this task because the CDC website provides a specific area for people to select their destination, along with specificities, to give them the updated report of what vaccines are necessary and what they’re at risk for.  Vaccines and side effects for babies o Children, especially newborns, are highly susceptible to diseases because of their weak immune system. We included this task because it is essential for mothers to have current and correct information about vaccines to protect their children.  How quickly an outbreak can spread and how to contain it o The United States is a popular place for international tourists to visit, some may not have the proper vaccinations and bring diseases with them, potentially affecting natives. We included this task because it is important to know how fast an outbreak may spread and how to contain it in case an outbreak plagues the nation.  Risk of radiation poisoning and safe levels of exposure o Radiation is found in common areas that most may not be aware of. We included this task because it is vital for people to know how to treat radiation poisoning, how to find out about different types of radiation in homes, and how much is too much exposure.  Injury statistics o Injuries happen daily and are sometimes fatal or nonfatal. We included this task because people should know how to prevent injuries or react when injuries occur.
  • 9. 9 Results Task Completion Success Rate Two participants experienced difficulty completing the fifth task. None of our participants spend a significant amount of time on the internet and are not experienced navigating through a medical web site. Due to this information, it proves that some tasks were too hard to find and users became frustrated and gave up or thought it took too much time and effort when they finally did completed the task. Table 3: Task completion rates Participant Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 1 Y Y Y Y N Y 2 Y Y Y Y Y Y 3 Y Y Y Y N Y # Of Success 3 3 3 3 1 3 Completion Rate 100% 100% 100% 100% 33.33% 100% Task Difficulty The following procedure was performed to test each task with each participant individually. The participants listened to the task read aloud by the facilitator, performed the task, and were asked to fill out the corresponding question in the post-test questionnaire. In the questionnaire, participants rated the level of difficult on a Likert scale and added comments.
  • 10. 10 1. Looking up information about parasitic diseases Each participant chose different ways to look up information about parasitic diseases. Participant #1 and #3 correctly clicked on the Diseases and Conditions tab, which easily led them to find information about different parasitic diseases. Participant #2 chose to use the search bar; the search results did not provide the best or most accurate answer, though still had useful information. Participant #3 commented that using the search bar did not “give me desired results.” Participant #2 whom used the search bar thought it “made it easy to find information.” The Likert results for this task are seen in the following pie chart. 2. Family traveling internationally and inquiring necessary vaccines and risks. All participants correctly selected Traveler’s Health and selected their destination, specificities, and received accurate information about vaccines and risks for international travel. 1 participant felt the selection boxes for the specifics made the search much easier. The Likert results for this task are seen in the following pie chart. 67% 33% 0%0%0% Task 2 Very Easy Easy Neither Difficult Very Difficult 0% 67% 33% 0%0% Task 1 Very Easy Easy Neither Difficult Very Difficult Figure 1: Task 1 Ratings Figure 2: Task 2 Ratings
  • 11. 11 3. Vaccines and side effects for babies As participants familiarized themselves with the site, they recognized which tabs would have what type of information. Participants #1 and #3 clicked on Healthy Living and selected Vaccines and Immunizations, which is what our team selected when performing the tasks prior to the test. It was somewhat confusing and difficult for them to find accurate information, but eventually both found some useful information. Surprisingly, Participant #2 found the chart our team found by using the search bar. The other two participants did not find that chart, which had the most detailed and prominent information. The Likert results for this task are seen in the following pie chart. 34% 33% 33% 0%0% Task 3 Very Easy Easy Neither Difficult Very Difficult Figure 3: Task 3 Ratings
  • 12. 12 4. How quickly an outbreak can spread and how to contain it Participants #2 and #3 did not click on the correct tabs and therefore, did not receive the best information about outbreaks. One participant had to scroll through news articles that did not relate to outbreaks in the U.S., only significant international plagues. It was unhelpful for their search. The other participant used the search bar and only received information about one specific outbreak. Overall, two of the three participants did not know how they felt about this task. The Likert results for this task are seen in the following pie chart. 0% 33% 67% 0%0% Task 4 Very Easy Easy Neither Difficult Very Difficult Figure 4: Task 4 Ratings
  • 13. 13 5. Risk of radiation poisoning and safe levels of exposure Participant #1 correctly clicked on the Emergency Preparedness tab. One participant clicked on the Healthy Living tab while the other used the search bar. Overall, none of the participants found the information that our team found, which provided accurate and a sufficient amount of information. Two out of the three participants found this task to be difficult or very difficult. On average, this task took the longest. The Likert results for this task are seen in the following pie chart. 6. Injury statistics Every participant correctly clicked on Diseases and Conditions, which led to a Data/Statistics tab. From there, each participant chose different statistical charts to look at, which was accepted because it was not specified. Each participant thought this task was easy to navigate and thought it was very straight forward. The Likert results for this task are seen in the following pie chart. 0% 100% 0%0%0% Task 6 Very Easy Easy Neither Difficult Very Difficult 0% 34% 0% 33% 33% Task 5 Very Easy Easy Neither Difficult Very Difficult Figure 5: Task 5 Ratings Figure 6: Task 6 Ratings
  • 14. 14 Time on Task The longest time a participant took on a task was 6 minutes and 25 seconds, which was Task 4. While participant #1 took 6 minutes and 25 seconds, the other two participants were able to complete between 4 to 5 minutes. Similarly, participant #1 took 4 minutes and 21 seconds to complete Task 6 while the other two participants took about half of that time to complete it. It appeared that each participants took more or less time on different tasks than the other participants, so none of the tasks took around the same time for all three participants. Another example of this can be seen in Task 1 in which one participant completed the task in less than two minutes while the other participants took between 2 to 4 minutes longer. Table 4: Time taken to complete each task Participant Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 1 235s 370s 205s 385s 353s 261s 2 310s 375s 247s 330s 221s 141s 3 110s 147s 243s 247s 368s 192s Total Average 218s 297s 232s 321s 314s 198s Errors Our group initially completed the tasks individually and wrote down exactly what we clicked. Based upon that information, each participant had a large amount of errors in almost every task. One participant relied solely on using the search bar for every task. Overall, in four out of the six tasks, 2/3 of participants made errors. In every task, there was at least 1 participant who made no errors. Other than Tasks 3 through 6, nobody made more than 1 error. Task #2 was completed without any errors by all three participants. Most errors resulted from clicking the wrong navigation bar tab. The amount of information on the different pages caused several errors. Some titles of articles were too long and specific that participants did not want to read them. Participants had to search closely to find enough information to complete the task. N/A means the participant was unable to complete the task.
  • 15. 15 Table 5: Number of errors in each task Participant Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 1 0 0 0 0 N/A 1 2 3 0 1 3 3 3 3 0 0 6 4 N/A 0 Data Summary Each participant selected “Easy” for three of the six tasks completed, therefore the usability test was easy overall for our participants. Each participants had difficult in different areas since they all used different methods to complete the specified task. As seen in Figure 1 in the Task Difficulty section above, Task 1 (Looking up information about Parasitic Diseases) had a 67% “easy” rating, and a 33.33% “Neither” rating. Task 2 had a 67% “Very Easy” rating compared to the 33.33% “Easy” rating, the participant that put “Easy” said the “topics were well outlined.” Task 3 had a variety of difficulty ratings. 34% (1 participant) selected “Very Easy”, 33% (1 participant) selected “Easy”, and 33% (1 participant) selected “Neither.” Task 4 had a 67% “Neither” rating and a 33% “Easy” rating. It would be accurate to conclude that many ratings have a two to one ratio of agreeance. Task 5 had another three-way split with 33% “Very Difficult”, 33% “Difficult” and 33% “Easy.” This task provided the most difficulty with our participants. Two of the three participants resulted with inconclusive evidence because they exhausted all their options and in the end, did not find the information they were looking for. There was a 100% success rate for Task 6. All participants noted that it was easy, straight forward, and it was easy to find.
  • 16. 16 Recommendations Based on the results of our usability test and the feedback we received from the participants we have numerous recommendations for the CDC.gov website. Our first suggestion regards the links in the navigation bar that can be found throughout the website. One of our test participants repeatedly found them difficult to click on. This leads us to the Diseases and Conditions page. Our participants found the list to be a little too comprehensive, and the video on the page did not help, as the participants didn’t like both that it played automatically and that it didn’t have any sound. A recourse to this might be to allow the user to show and hide sections of the page as they need it. As for the video, we would suggest getting rid of it and replacing it with an animated GIF, as there’s no point of a video without sound. Also, the Symptoms and Treatments pages should be merged, with the resultant page becoming the first page. Lastly, one participant wondered why they could only find scabies in the Parasites list. Participants, however, were able to find the information they were looking for. For the Traveler’s Health page, the drop-down list for this is too long. Considering the length of destinations, a drop-down list probably shouldn’t be used for this. Also, our testers wanted more information on the Traveler’s Health page. From our third scenario, we came to the conclusion that there should be a section for new mothers, as our participants had difficulty finding information for this. When our participants did find the information, they noticed that the “Before, During, and After” section only had information on what vaccines were needed before a pregnancy. We recommend that information on what vaccines would be necessary after a pregnancy should be added, along with information on which vaccines would be needed by infants. Our participants were able to quickly find information on an outbreak, however the presentation of this information could use some work. Conclusion For most of our tasks, our participants navigated the CDC.gov website easily. However, they were displeased with the presentation of this information. Among their complaints, they thought the navigation links needed to be easier to click on, they found information scattered in different places, and they didn’t find information on the pages they expected to find them on. In one case, they couldn’t find information on the healthy level of radiation. However, in most cases, the information was there, it was just poorly presented and scattered all over the place. With this in mind, we think that the best course of action for the CDC is reorganization and redesign the website so that it places the information people need where people would expect to find it.
  • 17. 17 Appendix Pre-Test Questionnaire Name (please print): ___________________________________________ 1. What is your gender? ❏ Male ❏ Female 2. What is your age? ________ 3. How many hours on average do you spend on the Internet each day? ❏ 0-2 hours ❏ 3-5 hours ❏ 6-8 hours ❏ Over 9 hours 4. Do you go online to find information about any of the following provided (check all that apply): ❏ Diseases ❏ Symptoms of illnesses ❏ Vaccines ❏ Safety Information ❏ Other ❏ Not Applicable 5. Which of the following science related courses have you completed? (check all that apply) ❏ Biology ❏ Anatomy & Physiology ❏ Microbiology ❏ Other ❏ Not Applicable 6. Have you visited any of the following websites? (check all that apply): ❏ mayoclinic.org ❏ cdc.gov ❏ niaid.nih.gov ❏ familydoctor.org ❏ fema.gov ❏ Other
  • 19. 19 Detailed Usability Script Hello and thank you for volunteering to help us out. My name is Juan Hernandez and I am here today with two colleagues, Jessica and Taylor. They will be taking notes as I guide you through this session. We are asking people to evaluate the user-friendliness of the Center for Disease Control’s website. I would like to take a moment to explain what you should expect. Before the evaluation, I will read you a scenario and you will navigate the site accordingly. There are six scenarios, but the evaluation should not take longer than thirty minutes. We are not evaluating your skills as the user, so do not worry about making mistakes. We are only evaluating how you navigate the site, and what your thoughts are about it. Your feedback today will help us evaluate this website. During the evaluation, I ask that you please think out loud. Hearing the way you perceive what you are seeing is important to our evaluation of the site. What we ask is that you say what you are looking at, what you are trying to do, and what you are thinking. I understand that thinking out loud seems strange, but it is important that we hear why you make the choices that you do, and what thoughts you have while you are navigating the site. Don’t worry about offending anyone in the room, we want to hear your honest thoughts. Since we are evaluating the site, not you, feel free to ask any questions out loud, but I may not answer them right away. We are trying to simulate normal circumstances in which people use this site. If you need to take a break at any point, please let me know. If we reach the end of the evaluation and you still have questions, I’ll try to answer them. Afterwards, you will complete a quick post-test survey and this session will be complete. Thank you again for your time. Do you have any questions before we begin? Let’s get started! Pre-test Questionnaire Before we begin, we would appreciate it if you took this pre-test questionnaire so that we will be able to see how much Internet and medical website experience you have. This will help evaluate the results by showing up your amount of experience. Thank you for filling out the questionnaire. During Testing It is important that I must remind all of you to think out loud and say why you are choosing to navigate the site as you are. I will present different tasks for all of you to solve using the site. When the tasks are completed we will pass out the post-test questionnaire, which is similar to the pre-test questionnaire that will help evaluate the results.
  • 20. 20 Post-test Questionnaire Thanks for all of your help; we are almost done with the evaluation. Here is the post-test questionnaire that I mentioned earlier. Please fill out the questionnaire so that we can evaluate today’s results. You will be marking the difficulty of each task, along with a brief explanation of why. After the Test Thank you for your help today. This information will be very helpful as we evaluate all the results that have been gathered, have a great day! Tasks 1. In your biology class, you heard about parasitic diseases, and you wanted to research it further. 2. Your neighbor, his wife, and two children are planning a trip to Europe and want to know what vaccines are needed and what diseases they’re at risk of. 3. Your friend Jane recently had a baby and wants to know the necessary vaccines and potential side effects. 4. You’re in your statistics class and start discussing a recent outbreak plaguing the nation. You want to find more information about how quickly an outbreak spreads and how quickly it can be contained. 5. You want to work as a radiation engineer and need to find information about the risk of radiation poisoning and a safe level of exposure. 6. You’re writing a paper for class about injury statistics, and want to find more information.
  • 21. 21 Post-Test Questionnaire Name (please print): ___________________________________________ 1. How easy or difficult was it to find information about a disease? Very Easy Easy Neither Difficult Very difficult Why was it easy or difficult? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 2. How easy or difficult was it to find information about vaccines and risks? Very Easy Easy Neither Difficult Very difficult Why was it easy or difficult? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 3. How easy or difficult was it to find information about vaccines and side effects for babies? Very Easy Easy Neither Difficult Very difficult Why was it easy or difficult? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________
  • 22. 22 4. How easy or difficult was it to find about an outbreak? Very Easy Easy Neither Difficult Very difficult Why was it easy or difficult? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 5. How easy or difficult was it to find information about radiation? Very Easy Easy Neither Difficult Very difficult Why was it easy or difficult? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 6. How easy or difficult was it to find information about injury statistics? Very Easy Easy Neither Difficult Very difficult Why was it easy or difficult? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________
  • 23. 23 Raw Usability Test Table 6: Participant pre-test responses Participant 1 Participant 2 Participant 3 Q1 Male Male Female Q2 20 23 22 Q3 0-2 hours 3-5 hours 0-2 hours Q4 Symptoms of Illnesses and Other Diseases, Symptoms of Illnesses, Vaccines, Safety Information, and Other Diseases, Symptoms of Illnesses, Vaccines, Safety Information, and Other Q5 Not Applicable Biology and Other Biology and Anatomy and Physiology Q6 Other Other Mayoclinic.org and CDC.gov Table 7: Participant Likert Responses Very Easy Easy Neither Difficult Very Difficult Q1 0 2 1 0 0 Q2 2 1 0 0 0 Q3 1 1 1 0 0 Q4 0 1 2 0 0 Q5 0 1 0 1 1 Q6 0 3 0 0 0
  • 24. 24 Table 8: Participant Comments Participant 1 Participant 2 Participant 3 Q1 “It has basic information so it could work alright.” “I don’t care where the sidebar links are but its location is unique.” “Why is Scabies the only kind of parasite listed?” Q2 “I mean yeah, most of this information seems pretty basic.” “I would like if they had a suggestion area a ‘help us help you’ recommendation section.” “I’m going to select immune compromised in case there are children or older people.” Q3 “The page should be bigger and more condensed.” “The hyperlink needs a different font because I wouldn’t know it was a hyperlink.” “It has information for before and during pregnancy but not after.” Q4 “I like the pamphlet it’s colorful, catches the eye, and easy to read.” “I wish they had links with the key words listed first.” “Oh this is talking about foodborne illnesses, this wasn’t what I was expecting.” Q5 “I have no idea how to answer this, I even typed in the search bar and still nothing.” “The picture gives off decent amount of information.” “This opened up a different website.” Q6 “There is no legend for the colors, I have no idea what this means.” “I think the chart, it says a lot.” “I don’t wanna fill out a form to get information.”
  • 25. 25 Participant #1 Pre-test 1. Male 2. 20 3. 0-2 hours 4. Symptoms of Illnesses, and Other 5. Not Applicable 6. Other Task #1 ● Clicked on Diseases and Conditions ● Clicked More ● Searched “J” for Japanese Encephalitis ● Clicked Symptoms ● Time: 3 minutes and 55 seconds ● Mistakes: None Task #2 ● Clicked on Traveler’s Health ● Clicked on Destinations ● Selected United Kingdom ● Selected Traveling with Kids ● Selected Pregnant ● Pressed Go ● Time: 6 minutes and 10 seconds ● Mistakes: None Task #3 ● Clicked Healthy Living ● Clicked Vaccines and Immunizations ● Typed “pregnant woman” in the search bar ● Selected Vaccine Information for Adults link ● Selected List of Vaccines ● Time: 3 minutes and 25 seconds ● Mistakes: None Task #4 ● Clicked Emergency Preparedness ● Clicked Recent Outbreaks ● Selected Illness ● Selected Infection Control ● Selected Ebola Outbreak ● Time: 6 minutes and 25 seconds ● Mistakes: None
  • 26. 26 Task #5 ● Clicked Emergency Preparedness ● Clicked Radiation Emergency ● Clicked Radiation Health ● Clicked Questions about Radiation ● Clicked Impact of Radiation in Daily Lives ● Clicked Learn more ● Clicked Information about Professionals ● Starts from the homepage ● Clicked Healthy Living ● Clicked More ● Typed “safe radiation exposure” in search bar ● Selected Radiation Emergencies link ● Clicked More Information ● Clicked Environmental Health ● Clicked Investigations ● Clicked General Radiation ● Time: 5 minutes and 53 seconds ● Mistakes: N/A Task #6 ● Clicked Healthy Living ● Clicked back to the homepage ● Clicked Diseases and Conditions ● Clicked Data/Statistics ● Clicked Injuries/Violence ● Selected Non-fatal Data ● Clicked on pictures ● Selected Leading Causes of Death by Age Groups ● Time: 4 minutes and 21 seconds ● Mistakes: 1 o Clicked on Healthy Living, realized it was wrong, and clicked on Diseases and Conditions
  • 27. 27 Post-test 1. Easy ○ Alphabetic list. 2. Very Easy ○ Quick search with selection boxes. 3. Easy ○ Searched for pregnant women under vaccines. 4. Neither ○ Obscure directory. 5. Very Difficult ○ Could not find the information I needed. 6. Easy ○ Injury button from homepage. Participant #2 Pre-test 1. Male 2. 23 3. 3-5 hours 4. Diseases, Symptoms of Illnesses, Vaccines, Safety information, and Other 5. Biology, and Other 6. Other Task #1 ● Typed “parasitic diseases” in search bar ● Clicked on the first link ● Selected “L” ● Selected CDC in Lebanon ● Clicked Travel ● Time: 5 minutes and 10 seconds ● Mistakes: 3 ○ Did not click on Diseases and Conditions, did not select More, and did not find information about parasitic diseases. Task #2 ● Clicked Traveler’s Health ● Clicked Destinations ● Selected Ireland ● Clicked Go ● Time: 6 minutes and 15 seconds ● Mistakes: 0
  • 28. 28 Task #3 ● Typed “newborn vaccine” in the search bar ● Selected For Everyone ● Selected Infants and Children ● Clicked 2015 Immunizations Recommended for Children Newborn to 6 Years Old ● Time: 4 minutes and 7 seconds ● Mistakes: 1 ○ Did not click on Healthy Living Task #4 ● Clicked Diseases and Conditions ● Typed “plague outbreaks” in the search bar ● Selected CDC Current Outbreak List ● Looked at the Breakdown of U.S. and International ● Selected Cyclospinosis-Parasites ● Time: 5 minutes and 30 sec ● Mistakes: 3 ○ Did not click on Emergency Preparedness, did not select Recent Outbreaks and Incidents, did not use the A-Z Index Task #5 ● Typed “radiation levels” in the search bar ● Selected Exposure/Levels ● Clicked U.S. Environmental Protection Agency link ● Clicked on the link under the picture ● Found a Relative Doses Chart ● Time: 3 minutes and 41 sec ● Mistakes: 3 ○ Did not click on Emergency Preparedness, did not select Radiation emergencies, and did not click on Radiation and Your Health Task #6 ● Clicked Emergency Preparedness ● Selected More CDC Topics ● Clicked Injury, Violence, and Safety ● Time: 2 minutes and 21 sec ● Mistakes: 3 ○ Did not click on Diseases and Conditions, did not select Data and Statistics, and did not select Injury Prevention and Control
  • 29. 29 Post-test 1. Easy ○ Search bar made it easy to find but menu options did not seem directly related. 2. Very Easy ○ Directly from home page, the info is easy to search. 3. Very Easy ○ Straight forward, search & info 4. Neither ○ Would prefer more visual for quick search information 5. Easy ○ Unexpected finding but info was easy to understand. 6. Easy ○ Straight forward in terms of searching Participant #3 Pre-test 1. Female 2. 22 3. 0-2 hours 4. Diseases, Symptoms of Illnesses, Vaccines, Safety Information, and Other 5. Biology and Anatomy and Physiology 6. Mayoclinic.org and CDC.gov Task #1 ● Clicked Diseases and Conditions ● Selected More ● Clicked Parasites-Scabies ● Clicked on related links ● Selected the A-Z Index for parasites in general ● Typed “about parasites” in the search bar ● Time: 1 minute and 50 seconds ● Mistakes: None Task #2 ● Clicked Traveler’s Health ● Clicked Destinations ● Clicked Travelers ● Selected England ● Selected Traveling with kids ● Selected Pregnant ● Selected Immune Compromised
  • 30. 30 ● Read Routine Vaccines ● Time: 2 minutes and 27 seconds ● Mistakes: None Task #3 ● Clicked Conditions and Diseases ● Went back to the homepage ● Clicked Healthy Living ● Clicked Vaccines/Immunizations ● Clicked Vaccine Side Effects and Safety ● Selected Pregnant Women ● Selected Infants and Toddlers ● Selected Vaccines for Pregnant Women ● Selected Vaccines Before, During, and After Pregnancy ● Clicked Adult Vaccines ● Clicked on “I” for Infants ● Time: 4 minutes and 3 seconds ● Mistakes: 6 ○ First clicked Conditions and Diseases, selected Pregnant Women, selected Vaccines for Pregnant Women, selected Vaccines Before, During, and After Pregnancy, selected Adult Vaccines, selected “I” for Infants. Task #4 ● Clicked Diseases and Conditions ● Selected More ● Selected More CDC Topics ● Goes back to the homepage ● Selected Emergency Preparedness ● Selected Recent Outbreak News Article ● Selected 2006 Outbreak ● Selected 2014 West Africa Ebola Article ● Typed “what to do in an outbreak” in the search bar ● Clicked on Foodborne Illnesses ● Typed “outbreak preparedness” in the search bar ● Clicked Water-Related Emergencies ● Clicked on Outbreak Response ● Time: 4 minutes and 7 seconds ● Mistakes: 4 ○ Did not click Emergency Preparedness, did not select Recent Outbreaks and Incidents, did not use the A-Z Index, and did not find information about responses to plagues.
  • 31. 31 Task #5 ● Clicked on Healthy Living ● Clicked on more ● Clicked on Specific populations ● Clicked on Workplace Safety Information ● Selected “R” for Radiation ● Selected Radiation Dose Reconstruction ● Typed “safe radiation levels” in the search bar ● Clicked back to the homepage ● Clicked Healthy Living ● Clicked More ● Clicked Workplace Safety Information ● Clicked Hazards/Exposure ● Clicked UV Radiation ● Clicked back to Hazards/Exposure ● Clicked back to Workplace Safety Information ● Clicked Safety and prevention ● Clicked Industries and Occupations ● Typed “safe nuclear radiation levels” in the search bar ● Clicked the first link about Japan ● Linked to OSHA.com ● Time: 6 minutes and 8 seconds ● Mistakes: Not Applicable Task #6 ● Clicked Diseases and Conditions ● Clicked Data/Statistics ● Clicked Injury/Violence ● Selected Fatal Injury Data ● Selected Injury Report 1999-2013 ● Clicked Leading Causes ● Filtered: Female, Years 2000-2000, Treated and Released ● Submitted Request ● Time: 3 minutes and 12 seconds ● Mistakes: None
  • 32. 32 Post-test 1. Neither ○ There was too much information to personally sort through. Using the search bar did not give me desired results- it gave me articles instead of general date. 2. Easy ○ Topics were well outlined 3. Neither ○ Information on babies was not closely correlated with the information on pregnant women 4. Easy ○ Menu options allowed easy access 5. Difficult ○ Website did not list nuclear radiation specifically- I ended up at a completely different website. 6. Easy ○ Menu options made it easy, but the data was very messy.