Measuring health literacy:Assessing Current Approaches Andrew Pleasant, Ph.D. Health Literacy and Research Director Canyon Ranch Institute Northern New Jersey Maternal / Child Health Consortium June 5, 2012 Bergen, NJ
DisclosuresThe following speaker has financial relationships with commercial interests to disclose:• Andrew Pleasant, Ph.D. • Is a member of the Merck Speaker’s Bureau on health literacy. • Time to Talk CARDIO is an educational program paid for by Merck and developed in partnership with the American Academy of Family Physicians Foundation, Canyon Ranch Institute and RIASWorks.
Summary slide: The current stateof health literacy measurement
Who here …• Has used any of the existing measures or screeners of health literacy?• How did that go?
Existing screeners and measures - 1• Rapid Estimate of Adult Literacy in Medicine (REALM) (Davis et al., 1991; Davis et al., 2006; Davis TC, 1993)• Test of Functional Health Literacy in Adults (TOFHLA) (Gong et al., 2007; Parker, Baker, Williams, & Nurss, 1995),• Health Activities Literacy Scale (HALS) (Rudd, Kirsch, & Yamamoto, 2004),• Newest Vital Sign (NVS) (Weiss et al., 2005),• Wide Range Achievement Test Fourth Edition (Dell, Harrold, & Dell, 2008),
Existing screeners and measures - 2• Stieglitz Informal Reading Assessment of Cancer Text (SIRACT) (Agre, et al., 2006),• Medical Achievement Reading Test (MART) (Hanson-Divers, 1997),• National Adult Reading Test (NART) (Uttl, 2002),• Literacy Assessment for Diabetes (LAD) (Nath, Sylvester, Yasek, & Gunel, 2001),• Nutrition Literacy Scale (NLS) (Diamond, 2007),• Short Assessment of Health Literacy for Spanish-speaking Adults (SAHLSA) (Lee, Bender, Ruiz, & Cho, 2006),
Existing screeners and measures - 3• an instrument targeting Canadian adolescents,• a “talking touchscreen” approach,• Demographic Assessment of Health Literacy (DAHL) (Hanchate, Ash, Gazmararian, Wolf, & Paasche-Orlow, 2008).• Items from the 2003 National Assessment of Adult Literacy (Baldi et al., 2009)• Health Literacy Skills Instrument (HLSI) (McCormack et al., 2010)• Mandarin Health Literacy Scale (MHLS) (Tsai, Lee, Tsai, & Kuo, 2011)
Existing screeners and measures - 4• The Agency for Healthcare Research and Quality (AHRQ) developed a “health literacy item set” for the Consumer Assessment of Healthcare Providers and Systems surveys ((AHRQ), 2007).• The Joint Commission is embarking on an effort to develop health literacy standards as part of its hospital accreditation process.• The Eurobarometer has recently completed a health literacy assessment in eight countries.
Most new measures are validatedagainst older measures – but …Measure Exact description General description N=207; convenience sample; 54% black; 76% Black women withREALM female; 42% dropped out of high school less education N=403; app. 20% refusal; 11% failed screening; Hispanic and convenience sample, 45% African American African AmericansTOFHLA “indigent”; 45% Hispanic; 58.5% less than with less high school graduate/GED. education N=500 (250 eng; 250 Spanish); 20% refusal;Newest mean age 41; 21.5% white, 73% Hispanic; Hispanic womenVital Sign 84 men; 416 womenChew’s N=332; 5% women; 81% white; 86% GED or single higher; ambulatory pre-op clinic (excluded White men with item ‘worst’ cases) GED or higher screenerWallace’s N=305; 68% female; 81.3% insured by White women with single TennCare/Medicare; only English speaking; less than high item 85.2% White; 88% less than high school school education screener education
Let’s take (part of) the TOFHLA!• Cloze method - multiple choice• Fyi, this isn’t the only design for the cloze method. Imagine the difference if there were NO choices (exact answer/ acceptable answer) .. Or you struck (X-ed) out an incorrect word and replaced it… or, score by difficulty of word? Etc.
Troubles with the TOFHLA: A brief example• Average refusal + exclusion = 40% (n=48)• No consistency in use or reporting: ○ Meta-analysis of data is NOT possible• No random samples of the general population• Meta-analysis population significantly different than U.S.• Both ceiling and floor effects• Several biases identified by researchers• Inconsistent data - linear or categorical
Newest Vital Sign: A pint of ice cream?• What is health literacy?• What is the most important question in this area of healthy behavior?
Chew / Wallace… final choice• How often do you have problems learning about your medical condition because of difficulty understanding written information? (note .. 15th – 17th grade level)• How confident are you filling out medical forms by yourself? (**Wallace) (10-12th)• How often do you have someone help you read hospital materials? (** Chew) (8-10th)
A fundamental distinction• The goal of screening … ○ divide people into healthy and sick categories (have/ have not). ○ In clinical contexts, this demands short, quick & easy to use• The goals of measurement … ○ advance knowledge - i.e. test hypotheses ○ explore and explain structure and function ○ monitor effectiveness and equity of interventions ○ indicate major problems confronting society ○ contribute to setting policy goals Equivalent to the difference between an “old-fashioned blood pressure cuff, stethoscopic, and manual abdominal health check-up and a comprehensive health examination” (Breslow, 2006)
Concerns about screening• SHAME … a silent barrier• Almost 40% of patients with low health literacy who also acknowledged they have trouble reading admitted shame.• Of those … ○ 67.2% had never told their spouses ○ 53.4% had never told their children• Nineteen percent (19%) had never disclosed their difficulty reading to anyone.
AMA Foundation says …• Screening/measurement is fine for research, but its not appropriate for daily clinical practice.• "Clinicians can better spend their time ensuring that all their patients understand the medical information they need to know to care for themselves."
Measuring what? Health Care Patients / System Public Ability to Level of Health participate demand Literacy (Reception (Sending skills) Skills) Nothing Some measures skills NAAL and readability tools; on either side in many new OSCEs context screeners
Critiques of existing screeners: A summaryExisting measures/screeners of health literacy:• are not designed to test or advance an underpinning theory of health literacy,• are limited in approach to evaluating skills - not behavior change or capacity to change (e.g., some overly rely on the cloze formatted reading test while others only evaluate word recognition and not understanding),• lack cultural sensitivity and can exhibit bias toward certain population groups,• do not measure an individual’s ability to prevent illness and injury,
Critiques of existing screeners: A summary• are not directly useful for informing or evaluating health promotion and communication interventions (e.g., a pre-post design), curricula, policy, or schemes to pay physicians based on performance,• place a problematic burden and potentially harmful label on patients being tested in clinical settings,• do not evaluate spoken communication skills,• do not consider health literacy as a public health issue,
Critiques of existing screeners: A summary• ambiguous item wording,• do not adequately distinguish between people at very low and very high levels of health literacy,• were not subjected to rigorous psychometric analysis,• have not been used in a consistent way,• focus on a single dimension while health literacy involves multiple dimensions,• may be biased toward those with recent experience with the health care system or content area, and
Critiques of existing screeners: A summary• the variations among the tools and how they have been used make it difficult to compare experiences or results across studies to definitively establish the relationship of health literacy to health status.
The NIFL (now LINCs) discussion:Deciding what’s needed• Over 200 messages• Approximately 80 contributors• At conclusion of week long discussion: ○ Created an online survey tool ○ 123 respondents ○ 4 day time frame
Possible consensus statements and responses Strongly Strongly Disagree Disagree Agree AgreeNew measures of health literacy need to be 1% 8% 38% 53%developed.New measures of health literacy need to be 0% 5% 44% 51%based on sound theory.Measurement of health literacy needs to be 0% 2% 38% 60%relevant to actual experiences.Existing measures of health literacy, whileimportant to the early development of the 2% 11% 44% 44%field, do not match the understanding ofhealth literacy that has developed.
Possible consensus statements and responses Strongly Strongly Disagree Disagree Agree AgreeWe need to be able to measure both sides ofthe health literacy equation - the health 1% 2% 27% 70%literacy of individuals and the health literacyof systems/health professionals.Health literacy measurement should not be 4% 21% 38% 38%prioritized in the clinical context.No single methodological tool is up to thetask of measuring health literacy, therefore ameasure of health literacy must incorporate 1% 8% 44% 48%multiple methodologies. This may includeboth quantitative and qualitativemethodologies.
Possible consensus statements and responses Strongly Strongly Disagree Disagree Agree AgreeA measure of health literacy needs to bevalidated with a broad population, not just a 0% 4% 35% 61%limited sample.A measure of health literacy should include 0% 20% 48% 32%evaluation of spoken language skills.A measure of health literacy will be multi-dimensional, addressing both multiple 1% 4% 44% 51%conceptual domains and multiple skills.
Possible consensus statements and responses Strongly Strongly Disagree Disagree Agree AgreeAs you cannot ‘see’ health literacy,the measure must sample from allthe conceptual domains outlined bythe underlying theory or conceptual 1% 8% 68% 24%framework. The measure can becomprehensive but does not have toinclude everything.A measure of health literacy thatfocuses solely on the clinical settingis inappropriate when researching 2% 11% 43% 45%public health behaviors andoutcomes.
What should be included in a measure of health literacy? Strongly Strongly Disagree Agree Disagree AgreeFinding/obtaining 0% 7% 43% 51%Understanding 0% 0% 22% 78%Evaluating/processing 0% 2% 34% 64%Communicating/ Being able to 0% 0% 35% 65%communicateUsing information 1% 2% 28% 69%Making informed choices 1% 5% 26% 68%Making appropriate choices 7% 17% 28% 49%
How should you build a new measure?• Explicitly built on a testable theory or conceptual framework of health literacy.• Multidimensional in content and methodology.• Measure on a continual, not a categorical basis.• Treat health literacy as a ‘‘latent construct.’’• Honor the principle of compatibility. ○ E.g. basing measurement of health literacy on an ice cream nutritional label is not compatible with a clinical setting.
How should you build a new measure?• Allow comparison to be commensurate across contexts including population groups, cultures, life courses, health topics, and research settings.• Prioritize social research and public health applications versus clinical screening.• Others you may suggest?
What I suggest you do now!• Treat health literacy as the way to shape the intervention – measure desired outcomes.• Use the Calgary Charter on Health literacy model to guide the design of your intervention. FIND EVALUATE COMMUNICATE USE UNDERSTAND
• Now it is your turn!What do you think should come next for health literacy measurement? What are you doing now?
Thank you! Andrew Pleasantandrew@canyonranchinstitute.org