The study found that users expect diseases to be categorized differently than clinical definitions. It recommends linking diseases from their expected categories to the clinical home category. Definitions should be provided for categories. Parents and non-parents had some differences in expectations, while web users and patient panelists differed more, suggesting intercepting web users for web behavior studies.

1. Conditions & Diseases Findability
- improving findability for up to 88% users
Kaitlan Chu
January 2012
UX | Digital Marketing | Cleveland Clinic Confidential 1

2. Contents
Agenda Page Number
Background 3-5
Pilot Study & Findings 6-12
Principal Study & Findings* 14-21
Executive Summary 22-
Appendices 28-
Categories & Definitions
Diseases & Definitions
UX | Digital Marketing | Cleveland Clinic Confidential 2

3. Background
UX | Digital Marketing | Cleveland Clinic Confidential 3

4. Objectives
• Cleveland Clinic Children’s Hospital wants to learn how
users mentally associate diseases with categories.
• Are they in line with the clinical definitions?
• E.g. by clinical definition, “Cyclic Vomiting Syndrome” belongs in the
category “Neurological Conditions”. Do users know to look there?
• Competitive Analysis shows there is no consistent
medical categories among Children’s Hospital’s
competitor websites
• User groups: Is there difference in how different user
groups search for health info?
• Parents v.s. Non-parents
• Web users v.s. Cleveland Clinic Patient Panelists
• How many % of web surfers are medical professionals?
UX | Digital Marketing | Cleveland Clinic Confidential 4

5. Diseases & Conditions Accordions
UX | Digital Marketing | Cleveland Clinic Confidential 5

6. Methodology: Closed-card Sort
• Participants are provided with a predetermined set of medical category names.
i.e. Neurological Conditions
• They then place the disease cards to these categories.
i.e. Pink eye
• A total of 45 cards, 19
categories
• All come with definitions
• Each participant was
presented with 10 random
disease cards
• Each participant sees all the
medical categories, displayed
in alphabetical order
UX | Digital Marketing | Cleveland Clinic Confidential 6

7. Pilot Study – Qualitative
UX | Digital Marketing | Cleveland Clinic Confidential 7

8. Pilot Study
• In-person card sort with 10
participants
– Melissa
– Lisa (minor)
– Jamil
– Judith
– Cheryl
– Linda (volunteers at the center)
– Kate
– Amy
– Bob (volunteer at the center)
– Ben
• Recruited from Ronald Donald
Family Room in Children’s
Hospital
• Conducted in the lobby on 3rd
floor
• October 26-31, 2011
UX | Digital Marketing | Cleveland Clinic Confidential 8

9. Pilot Study
Sign-up Sheet Flyers
UX | Digital Marketing | Cleveland Clinic Confidential 9

10. Findings: Content (Pilot Study)
• Recommend – Provide definitions to Categories:
 Rheumatology is the category least participants are familiar with (8/10).
 Followed by Endocrinology, Orthopedics, and Pulmonary
 Recommend - Adding Categories:
 Most wished-for: "Sleep Disorders"
 “Urology” (in lieu of, split between Kidney and Infectious Diseases)
 “Therapy”
 “Gastroenterology” (in lieu of, Digestive Diseases was chosen)
UX | Digital Marketing | Cleveland Clinic Confidential 10

11. Findings: Content (Pilot Study)
 Diseases - difficult to place, require further attention
• Stuttering
• Most participants struggled with this card
• SID
• Split between Birth Defects and Pulmonary
• Sleep Disorders, Narcolepsy, Insomnia:
• Some stated they wished there is a "sleep disorders" category. In
lieu of, they're split among Neurological, Common Childhood
Illnesses, and Emotions & Behaviors*
 Diseases - no further study necessary
• Anemia: 100% agreement in placement of "Blood Disorders"
• Hearing Impairment: 89% agreement in Ear, Nose, Throat
* In line with quantitative results
UX | Digital Marketing | Cleveland Clinic Confidential 11

12. Findings: Web interface – the Accordions (Pilot Study)
Based on findings, definitions of medical categories should be provided on the
Diseases & Conditions accordions
Recommended Solutions:
Showing definitions in form of:
3. Rollover (Note: More than
half of participants didn't
notice the roll-over in the
card sort software)
5. Texts in smaller font size,
next to the Category title
7. Question Mark icon: on
click, show definitions
UX | Digital Marketing | Cleveland Clinic Confidential 12

13. Findings: Web interface – Screenshot (Pilot Study)
Screenshots too realistic?
Users tried to interact with the screenshot, as seen in multiple studies.
Possible solutions?
When using animated gif in instructions there was no problem.
Observation from this study Observation from last study, on .org
UX | Digital Marketing | Cleveland Clinic Confidential 13

14. Principal Study - Quantitative
UX | Digital Marketing | Cleveland Clinic Confidential 14

15. Study Fielding
• Study invitation appears on Children’s Hospital website
• Study invitation appears once per IP address
• There was no multiple entry (per email address)
• A total of 371 responses:
• 124 Children’s Hospital Web Users (Dec. 1 - 19, 2011. Avg. time taken 5.6 minutes)
• 247 CC Patient Panelists (Dec. 7 – 19, 2011. Avg. time taken 7.2 minutes)
UX | Digital Marketing | Cleveland Clinic Confidential 15

16. Participants’ Expectations Differ from Clinically Correct Categories
Disease Category
*Statistically significant,
95% confidence level Clinically-correct Participant-chosen Failure Rate
Croup Pulmonary Common Childhood Illnesses 64.9%
Cyclic Vomiting Syndrome Neurological Conditions 78.4%
Digestive Disorders
Head Lice Infectious disease Common Childhood Illnesses 89.0%
Dermatology
Hydrocele Urology Birth Defects 74.0%
Common Childhood Illnesses
Insomnia in Children Neurological Conditions Common Childhood Illnesses 76.0%
Emotions & Behavior
Lead Poisoning Blood disorder Neurological Conditions 40.2%
Pink eye Infectious disease Eye Care 82.0%
Common Childhood Illnesses
Spine Tumors Neurological Conditions Cancer 71.0%
Orthopaedics
Stuttering Neurological Conditions Emotions & Behavior 88.0%
Ear, Nose & Throat
Common Childhood Illnesses
UX | Digital Marketing | Cleveland Clinic Confidential 16

17. Participants’ Expectations Differ* from Clinically Correct Categories
Disease Category
Participant-chosen %* Clinically Correct
Croup Common Childhood Illnesses 64.9% pulmonary
Pulmonary 24.5%
Cyclic Vomiting Syndrome Digestive Disorders 78.4% Neuro
Head Lice Common Childhood Illnesses 48.4% infectious disease
Dermatology 40.9%
Infectious Diseases 10.8%
Hydrocele Birth Defects 56.5% Urology
Common Childhood Illnesses 17.4%
Insomnia in Children Common Childhood Illnesses 38.8% Neuro
*Statistically significant, Emotions & Behavior 37.6%
95% confidence level
Neurological Conditions 17.6%
Lead Poisoning Neurological Conditions 40.2% blood disorder
Blood Disorders 33.0%
Pink eye Eye Care 70.5% infectious disease
Infectious Diseases 12.8%
Common Childhood Illnesses 11.5%
Spine Tumors Cancer 42.0% Neuro
Orthopaedics 30.9%
Neurological Conditions 18.5%
Stuttering Emotions & Behavior 64.0% neuro
Ear, Nose & Throat 15.1%
Neurological Conditions 10.5%
Common Childhood Digital Marketing |9.3%
UX | Illnesses Cleveland Clinic Confidential 17

18. Diseases & Conditions Accordions
UX | Digital Marketing | Cleveland Clinic Confidential 18

19. When searching for health info, which groups behave more alike?
Parents / non-parents, or
Web users / panelists?
UX | Digital Marketing | Cleveland Clinic Confidential 19

20. Differences* (6/45 diseases) in Parents v.s. Non Parents
Parents v.s. non-parents have significant Disease Category Non-parent / Parent
different viewpoints in these categories
Bone Tumor
Non-significant categories Cancer 69% / 85%
Orthopaedics 27% / 8%
Cellulitis
Dermatology 79% / 73%
Infectious Diseases 0 / 14%
Diaper Rash
Common Childhood Illnesses 69% / 44%
*Statistically significant, Dermatology 28% / 46%
95% confidence level
Hand, Foot and Mouth
Common Childhood Illnesses 35% / 49%
Disease
Infectious Diseases 50% / 31%
Sore Throat
Ear, Nose & Throat 83% / 59%
Common Childhood Illnesses 13% / 37%
Testicular and Kidney
Cancer 70% / 80%
Cancer
Kidney 30% / 15%
UX | Digital Marketing | Cleveland Clinic Confidential 20

21. Differences* (12/45 diseases) in Web Users v.s. Panelists
Disease Category Panel / Web
Constipation Digestive Disorders 93% / 77%
Head Lice Common Childhood Illnesses 40% / 61%
Web Users v.s. Panelists have significant Dermatology 46% / 33%
different viewpoints in these categories Insomnia in Children Common Childhood Illnesses 46% / 25%
Non-significant categories Emotions & Behavior 39% / 36%
Neurological Conditions 9% / 36%
Narcolepsy in Children Neurological Conditions 49% / 75%
Pink Eye Infectious Diseases 18% / 3%
Common Childhood Illnesses 6% / 21%
Pituitary Tumors in Children Cancer 6% / 26%
*Statistically significant,
95% confidence level Restless Legs Syndrome in Neurological Conditions 58% / 37%
Children and Adolescents
Orthopaedics 6% / 19%
Respiratory syncytial virus Pulmonary 50% / 74%
(RSV) in your Child
Common Childhood Illnesses 28% / 11%
SIDS (Sudden Infant Death Birth Defects 31% / 56%
Syndrome)
Common Childhood Illnesses 32% / 12%
Sleep Disorders Neurological Conditions 24% / 44%
Common Childhood Illnesses 33% / 22%
Sore Throat Ear, Nose & Throat 76% / 58%
Common Childhood Illnesses 19% / 39%
Spine Tumors Neurological Conditions 13% / 33%
UX | Digital Marketing | Cleveland Clinic Confidential 21

22. Research Question
When searching for health info, which groups behave more alike?
Parents / non-parents 
Web users / panelists
WHY???
Attempt to answer with demographic data
UX | Digital Marketing | Cleveland Clinic Confidential 22

23. Demographic: Non-parent v.s. Parent
80.00%
Parent
30.00%
Age
70.00%
25.00%
60.00%
20.00%
50.00%
Non-parent
Non-Parent 15.00%
40.00% Parent
Parent
30.00% 10.00%
20.00%
5.00%
10.00%
0.00%
0.00% Under 18-24 25-35 36-45 46-55 56-65 66-75 76 or
Female M ale 18 years years years years years years above
35.00% 70.00% Medical Background
Education
30.00% 60.00%
25.00% 50.00%
40.00%
20.00% Non-parent
Non-parent
30.00% Parent
Parent
15.00%
20.00%
10.00%
10.00%
5.00%
0.00%
0.00% Yes, as a Yes, as a Yes, as a Yes, in Yes, in a No – I do
Graduat e Some Graduat ed Some Graduat ed Some high physician nurse medical some non- not work
degree/ post - college college high school or student other clinical in t he
PhD/ graduat e school less clinical role medical
Doctorat e work role f ield
128 Non-parents 241 Parents
UX | Digital Marketing | Cleveland Clinic Confidential 23

24. Demographic: Panelists v.s. Web Users
80.00% 40.00%
Parent Age
70.00% 35.00%
60.00% 30.00%
25.00%
50.00%
Panel
Panel 20.00%
40.00% Web
Web
15.00%
30.00%
10.00%
20.00%
5.00%
10.00%
0.00%
0.00% Under 18-24 25-35 36-45 46-55 56-65 66-75 76 or
Non-Parents Total Parents Total 18 years years years years years years above
35.00% 80.00% Medical Background
Education
30.00% 70.00%
25.00%
60.00%
50.00%
20.00% Panel Panel
40.00%
15.00% Web Web
30.00%
10.00%
20.00%
5.00% 10.00%
0.00% 0.00%
Graduate
Graduated
Graduated
Some high
graduate
college
No – I do
Some
school or
physician
Yes, as a
Yes, as a
Yes, as a
Yes, in
Yes, in a
degree/
not work
some
medical
Some
other
clinical
student
college
school
PhD/
post-
in the
nurse
non-
high
less
247 Panelists 124 Children’s Hospital Web Users
UX | Digital Marketing | Cleveland Clinic Confidential 24

25. How many % of web users are medical professionals?
UX | Digital Marketing | Cleveland Clinic Confidential 25

26. Summary
UX | Digital Marketing | Cleveland Clinic Confidential 26

27. Summary
• Each disease should reside in the clinical home, but links (in accordions) from
its participant-chosen significant categories.*
• The category accordions should include definitions of each category
• “Lungs, Breathing, & Allergies” to define “Pulmonary”
• The “Specialties” section on .org should leverage the findings of this study:
• Include category definitions
• Cross reference when appropriate
• Be mindful when using realistic screenshots in tutorials
• 60% of web users are medical professionals on Children’s Hospital site
• When conducting a study about web behaviors, consider intercepting web users
instead of patient panelists.
• Results were twice the difference compared to parent v.s. non-parent
• Consider including non-parent & minors in future studies:
• Children can be the dominating computer users at home, hence qualify as target users.
• Parent v.s. non-parent: despite the age difference, how they find health information is
relatively similar.
* See complete list UX | Digital Marketing | Cleveland Clinic Confidential 27

28. Diseases & Conditions Accordions
78.5% traffic coming from
Cyclic Vomiting Syndrome
Diseases-conditions /neurological/cyclic-vomiting-syndrome.aspx
Clinically correct
UX | Digital Marketing | Cleveland Clinic Confidential 28

29. Future Studies
UX | Digital Marketing | Cleveland Clinic Confidential 29

30. Limitations of this study & Recommendation to future studies
Tree Jack
• This study assumes “how people file
information away” is “how people
retrieve information”.
• Future studies should consider:
1. tree-jack instead of closed-card
sort. This study couldn’t, due to
the vendor’s software limitation
on number of cards shown to
each participant.
2. including a “I don’t know”
category.
UX | Digital Marketing | Cleveland Clinic Confidential 30

31. Limitations of this study & Recommendation to future studies
Observation during the pilot test (on screen 1280)
1. If users are on 1024x768 screen, (the dominating group of the traffic),
once they start putting cards in categories, the page becomes longer,
and users lost track of the categories below the fold. Hence skew the
result.
2. Participants have an (compulsive?) urge to fill each category with at
least one card. Hence skew the result.
3. Participants paid more attention to the categories on the top left. When
debating between two categories, they tend to pick the one closer to
the top left (shorter distance of drag and drop). Hence skew the result.
UX | Digital Marketing | Cleveland Clinic Confidential 31

32. Thank you
Please contact Kaitlan Chu if you have
any question regarding this study
chuk@ccf.org
216-448-1015
Special thanks to Sunny Lu, Zhijie Li
(Market Research)
UX | Digital Marketing | Cleveland Clinic Confidential 32

33. Appendices
UX | Digital Marketing | Cleveland Clinic Confidential 33