Julie Bershadsky's presentation from TAI 2014

1. DATA AND ALL THAT FUN
STUFF
Julie Bershadsky, PhD
Human Services Research Institute
2014 ITACC TA Institute

2. Data - surveys, questionnaires,
phone interviews, administrative
data, etc…

3. Data – surveys, questionnaires,
phone interviews, administrative
data, etc…

4. BUT…
• Data are important!
• Measure the right things
– Why, what, when
– How
• Why measure?
– Anecdotal evidence?
– Required
• Policy makers want to see it
– Evidence of change
• Baseline
• Improvement
– Accountability

5. What to measure?
• Process vs. outcomes
• Outcomes:
– Reflect goals
– Address issues that can be influenced
– Directional
– Accurate, valid and reliable
– Affordable to collect
– Culturally competent
– Interpretable
– Measurable!
• When to measure
– How often?

6. How to measure?
• Mode
– In-person
– Phone
– Mail
– Web
• Response options
• Sample?
– Who?

7. Using the data
• Do nothing
• Can compare to an internal standard or goal
• Can compare to external standard or goal
• Can compare to others
• Last two: complicated…..

8. The Complication
?

9. If things were simple:
is better than

10. But things are not simple
?

11. Hmmmm…..
?

12. Solution: Risk adjustment

13. Aha!

14. Still complicated…
• What should be adjusted?
• What to adjust for?
– Differ between GA and MA
• E.g. number of nice people
• E.g. age
– Affect the outcome
– BUT: legitimately affect the outcome (e.g.
biological factors)

15. National Core Indicators (NCI)
• Multi-state collaboration of state DD agencies interested in
measuring how well public systems for people with developmental
disabilities perform along several areas, including: employment,
community inclusion, choice, rights, and health and safety
• HSRI-NASDDDS collaboration
• Started in 1997
• Now in over 40 states
• Why:
– Allow state-to-state comparisons of performance of state DD
systems
– Influence national and state policy
– Inform strategic planning and priority setting at state and
national levels

16. National Core Indicators (NCI)
• What:
– Indicator-based, focus on outcomes and quality of life
– Individual characteristics of people receiving services
– The locations where people live
– The activities they engage in during the day including
whether they are working
– The nature of their experiences with the supports that they
receive (e.g., with case managers, ability to make choices,
self-direction)
– The context of their lives – friends, community
involvement, safety
– Health and well-being, access to healthcare

17. National Core Indicators (NCI)
• When:
– Yearly cycle
• How (NCI Adult Consumer Survey):
– Standardized face-to-face interview
– Response options: simple
– Sample in each state
• No a priori exclusion criteria, all 18+ adults with IDD
receiving at least one service in addition to case
management

18. National Core Indicators (NCI)
Using the data
• Reporting reqs
• Track improvement
• Comparing across groups
• Comparing with national “average”
• Comparing with other individual states
• Formulating policy
• Identify areas and priorities for QI
• Raising awareness

19. 0% 5% 10% 15% 20% 25% 30% 35%
Specialized Institutional facility
Group Home
Apartment Program
Independent Home/ Apt
Parent or Relative's home
Foster care/ Host home
Nursing Facility
Other
Don't know
6%
27%
4%
17%
34%
6%
1%
4%
0%
Residence

20. At Least One Psychotropic Medication by
Living Arrangement
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
institution community-based
residence
independent
home/apt
parents/relatives
home
53%
68%
53%
36%
20
Psych meds

21. Basic Exams & Screenings
• Higher
percentages in
provider-based
settings
• Lowest for
people living in
parent/relative
home
• Similar trend
across
indicators
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
institution community-based
residence
independent
home/apt
parents/relatives
home
94%
97%
91%
85%
95%
88%
73%
69%68%
69%
58%
43%
90%
68%
49%
46%
Physical
exam
Dental visit
Vision
screening
Hearing test
(5 years)
21

22. Choice
0%
10%
20%
30%
40%
50%
60%
70%
80%
50%
45%
64%
80%
60% 60% 58% 59%
22

23. Wages (‘09-’10)

24. Table 12.4 Proportion of people who reported they have enough privacy at home
State Overall In
State
N Institution Community-
Based
Individual
Home
Parent’s
Home
Significantly Above Average
ME 97% 220 n/a 98% 100% n/a
AZ 97% 311 n/a 95% 95% 98%
KY 95% 319 n/a 94% 100% 98%
GA 95% 381 n/a 96% 100% 93%
PA 94% 835 n/a 95% 96% 95%
Within Average Range
CT 93% 247 n/a 92% 95% 94%
MEORC 93% 279 n/a 92% 94% 93%
LA 93% 258 n/a 87% 95% 95%
OH 91% 345 n/a 89% 88% 94%
NY 91% 1,637 n/a 89% 93% 93%
MA 91% 379 n/a 90% 93% 91%
NC 91% 499 n/a 90% 94% 91%
AR 90% 226 90% 80% 93% 93%
MI 89% 244 n/a 81% 88% 94%
AL 88% 226 n/a 90% n/a 86%
MO 87% 341 n/a 79% 92% n/a
IL 87% 231 n/a 85% n/a 88%
Significantly Below Average
HI 82% 201 n/a n/a n/a 81%
NJ 81% 338 n/a 80% n/a 83%
SC 79% 300 n/a 76% 86% 77%
NCI Average 90% 7,817 90% 88% 94% 91%
Unadjusted:

25. Table 4.1 Number of times people reported they went out shopping in the past month
(Adjusted Variable)
State Adjusted
Average
N (observed)
HI 5.8 430
ME 5.8 317
NC 5.3 883
AZ 4.5 351
MI 4.3 408
NY 4.3 2,504
MO 4.2 497
LA 4.2 417
MA 4.2 494
PA 4.1 1,234
CT 4.0 346
GA 3.8 485
AR 3.6 387
AL 3.2 345
IL 2.9 333
MEORC 2.8 307
KY 2.6 471
OH 2.5 273
SC 2.5 402
NJ 2.5 409
NCI Average 3.9 11,293
Adjusted:

26. Example: Georgia
• Use of psychotropic medications above
average
• Further study:
– After transition from institutions
• Prescribed for the first time
• Number of prescribed meds wend up
• Initiative to reduce use
• Reflected in data

27. Example: Kentucky
• Behind other states in employment, health
and exercises, psych meds, loneliness and
friendship
• Employment: revised waiver, increased
payment for supported employment,
decreased payment for day activity services
• Psych meds: revised waiver
• New service: community access
• Targeted health and wellness training

28. Example: Arizona
• Identified deficits in
women’s healthcare
outcomes
• Took action to improve
women’s ob/gyn access
resulting in rate of
annual visits increasing
from 30% to 70%
– Educating parents
– Educating doctors
Prioritizes actions and
quality efforts on:
• Aging caregivers
• Well-woman checkups
• Loneliness & self advocacy
• Choice of case managers
• Exercise
• Service Costs
• Physician access

29. Example: Alabama
• Choice and decision-making identified as area
for improvement;
• Five year plan to increase # of PCP facilitators and offering SRV training
to families
• Using consumer and family surveys to
measure satisfaction pre- and post-
community placement from closing of state
facility

30. Why Are Data Important?
Without Data
• Could not distinguish policy
goals and actual outcomes
of those policies
• Could not compare the
effectiveness and outcomes
of specific services
• Could not track important
system changes – negative
or positive
• Could not track the
impact of system reform
With Data
• Can measure and develop
strategic goals
• Can enhance transparency
• Can involve individuals and
families in the
interpretation of results
• Can communicate system
values – e.g., choice, health,
relationships
NCI CA Movers Presentation

31. THE END

32. CONTACT INFO
JULIE BERSHADSKY, PhD
JBERSHADSKY@HSRI.ORG
617-876-0426 ext 2331