2. Research Team
• Principal Investigator: Frances Wen, PhD
• Co-Investigators:
Jagan Bokka, MD
Bhupinder Walia, MD
Michelle Farabough, MSKM
Danny Stout, MHR
Anna Jaffe, BS
3. Introduction
• An estimated 20% of population will be
65 or older by 2030.
• Alzheimer’s in US expected to triple over
next 50 years.
4. • Demand for information on how to navigate
and cope with issues faced in later life is
expected to increase.
• “Alzheimer’s, dementia, and memory loss”
among the top health topics searched online.
5. Health Literacy
“A patient’s ability to obtain, process, and
understand basic health information and
services needed to make appropriate health
decisions.” – AMA
– Gaining attention
last 10 yrs…
vulnerable population
– Important part of
health outcomes
6. Aim
“Evaluate the readability and suitability of
education materials found on 15 frequently
accessed health care websites targeted at
individuals seeking information about
Alzheimer’s, dementia, and memory loss.”
7. Website Selection
STEP ONE: “Page Rank”
• Two most widely used Internet search
engines Google and Yahoo
• Search terms “Alzheimer’s” “dementia”
and “memory loss”
• Top 15 websites for
each term on each
search engine
9. Websites Evaluated
• Governmental: NIH
• Nonprofit: Alzheimer’s Association
• Popular Medical: WebMD, Mayo Clinic,
Medicine Net, Everyday Health
• Others: Wikipedia, Health NY Times,
Real Age
10. Webpage Selection
STEP TWO
• Added search term “education” to search
terms for each website
• Identified unique url
“jumping off” point
12. Evaluation Methods
1. SAM: Suitability Assessment of Materials
an instrument used by health care
professionals to assess six main
categories:
– Content
– Literacy demand: SMOG
– Graphics
– Layout/typography
– Learning Stimulation
– Motivation
13. 1a. SMOG: Standard Measure of Gobbledegook
a commonly used literacy evaluation tool
instrument to assess reading levels
– Figured into SAM “Literacy demand”
14. 2. DISCERN
an instrument used by health care
professionals to assess consumer health
information relative to…
– Reliability (authority*) and whether or not you
should consider it a trusted source
– Specific details about the treatment choices
– Overall quality rating of the media
*more about this at the end of the presentation
15. 3. QUIS: Questionnaire for User Interface
Satisfaction
an instrument used to measure overall reaction,
overall website, terminology and information,
learning, and website capabilities
16. Procedure
• Two investigators independently
reviewed/assessed materials using SAM,
DISCERN and QUIS to evaluate suitability.
• Another investigator used SMOG to evaluate
readability and incorporate results in SAM.
• Interrater agreement analysis and descriptive
analyses were conducted.
17. Interrater Agreement: SAM
• Interrater agreement = .78
• < .40: No agreement
• .40-.59: Moderate agreement
• .60-.79: Substantial agreement
• >.80: Outstanding agreement
• Raters agreed on the SAM so the results could
be averaged between raters.
18. Results: SAM Adequate
1.6
1.37 1.36
1.4
1.2 1.23
1.17
1.2 1.1
1
0.8
0.6
0.4
0.2
0
Content Literacy Demand Graphics Layout and Typography
Learning Stimulation, Motivation
Cultural Appropriatene
• SAM scores: 0=Not Suitable, 1=Adequate, 2=Superior
• SMOG scores (an item on the Literacy Demand scale) were all
determined to be at or above the 10th grade reading level.
19. Interrater Agreement: DISCERN
• Interrater agreement = .48
• < .40: No agreement
• .40-.59: Moderate agreement
• .60-.79: Substantial agreement
• >.80: Outstanding agreement
• Raters did not agree very well on their scoring
of the DISCERN, so their ratings had to be
analyzed separately.
20. Results: DISCERN Above Average
4.4
4.2 4.03
4
3.8 3.68
3.54 3.6
3.6 3.46
3.4 3.33
3.2
3
Is the publication Quality of Overall rating
reliable information on
treatment choices
Rater 1 Rater 2
DISCERN scores were slightly above average for both raters.
On average, Rater 1 scored websites slightly higher than Rater 2.
21. Interrater Agreement: QUIS
• Interrater agreement = .73
• < .40: No agreement
• .40-.59: Moderate agreement
• .60-.79: Substantial agreement
• >.80: Outstanding agreement
• Raters agreed on the QUIS so the results could
be averaged between raters.
22. Results: QUIS Above Average
8 7.27 7.55
7 6.72 6.67
6.21
6
5
4
3
2
1
0
Overall Website Terminology Learning Website
reaction and website capabilities
information
• QUIS scores 0-9; higher scores represent higher user friendliness
• The average website scored above average on website usability.
23. Discussion
• 15 educational websites with information
about age-related cognitive impairment
evaluated. Overall the suitability and
readability of these materials are adequate.
• Future direction: User analysis
• Research focused on health literacy is critical
to improve health care outcomes.
25. “Authority”
– Date of creation/update
– History/longevity of publication or website
– Author or source: academic, government
– “Facework”: links or citations
– Accuracy of information
– Objectivity and coverage: how in-depth
