Book Paid Powai Call Girls Mumbai 𖠋 9930245274 𖠋Low Budget Full Independent H...
John Billings: Applying predictive risk approaches and models effectively
1. APPLYING PREDICTIVE RISK APPROACHES
AND MODELS EFFECTIVELY
June, 2012
New York University
Robert F. Wagner Graduate School of Public Service
2. WHAT I’M GOING TO TALK ABOUT
• What not to do
• What to try to do
• An example of how we almost got it right, but in the
end, not so much
3. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
4. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
5. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
– Intervention design flaws
6. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
– Intervention design flaws
– Intervention implementation flaws
7. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
– Intervention design flaws
– Intervention implementation flaws
• Don’t do it the way you do it in the U.K. [With noteable exceptions]
8. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
– Intervention design flaws
– Intervention implementation flaws
• Don’t do it the way you do it in the U.K. [With noteable exceptions]
– Model development limitations
9. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
– Intervention design flaws
– Intervention implementation flaws
• Don’t do it the way you do it in the U.K. [With noteable exceptions]
– Model development limitations
– Intervention design flaws
10. WHAT NOT TO DO
• Don’t do it the way we do it in the U.S.
– Model development limitations
– Intervention design flaws
– Intervention implementation flaws
• Don’t do it the way you do it in the U.K. [With noteable exceptions]
– Model development limitations
– Intervention design flaws
– Intervention implementation flaws
12. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
13. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
14. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
15. CASE FINDING ALGORITHM
RESULTS FROM A MULTI-HOSPITAL SYSTEM
USING ITS OWN DATA
Predicted Predicted Predicted
Non-Adm Adm Total
Actual - Non Adm 105,495 1,860 107,355
Actual - Adm 18,459 2,909 21,368
Actual - Total 123,954 4,769 128,723
Specificity 0.983
Sensitivity 0.136
PPV 0.610
20. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
21. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
22. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
23. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
– Recognize you are probably going to need more information
24. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
– Recognize you are probably going to need more information
– Get the incentives right
25. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
– Recognize you are probably going to need more information
– Get the incentives right
• Intervention implementation flaws
– Roll it out in at least quasi-experimental mode
26. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
– Recognize you are probably going to need more information
– Get the incentives right
• Intervention implementation flaws
– Roll it out in at least quasi-experimental mode
– Track “dosage” levels (who does what to whom and how)
27. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
– Recognize you are probably going to need more information
– Get the incentives right
• Intervention implementation flaws
– Roll it out in at least quasi-experimental mode
– Track “dosage” levels (who does what to whom and how)
– Avoid enrollment criteria “leakage”
28. WHAT TO DO
• Model development limitations
– Predict risks of expensive things you think you do something about
– Make sure your data base has most of the key risk factors
– Recognize the trade-offs between model accuracy and sensitivity
• Intervention design flaws
– Design the intervention after the risk model has been developed
– Use data from model development to help design the intervention
– Recognize you are probably going to need more information
– Get the incentives right
• Intervention implementation flaws
– Roll it out in at least quasi-experimental mode
– Track “dosage” levels (who does what to whom and how)
– Avoid enrollment criteria “leakage”
– Evaluate impact of the intervention as rigorously as possible
30. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
31. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
32. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
33. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
THEN GET SOME SMART PEOPLE IN THE ROOM
AND DESIGN THE INTERVENTION
34. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
Step 3: Implement/evaluate pilot projects based on this
information
- Pilot with a quasi-experimental design (intervention/control)
- Conduct “formative” evaluation during early phases of
implementation
- Assess impact of intervention on outcomes/utilization
35. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
Step 3: Implement/evaluate pilot projects based on this
information
- Pilot with a quasi-experimental design (intervention/control)
- Conduct “formative” evaluation during early phases of
implementation
- Assess impact of intervention on outcomes/utilization
Step 4: Disseminate results/Scale it up if it works
36. A SOMEWHAT IDEALIZED
SUGGESTED APPROACH TO PREDICTIVE RISK
MODELING AND EFFECTIVE IMPLEMENTATION
Step 1: See if you can develop a predictive model to identify patients
Evidenced-based management/policy making
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
Step 3: Implement/evaluate pilot projects based on this
information
- Pilot with a quasi-experimental design (intervention/control)
- Conduct “formative” evaluation during early phases of
implementation
- Assess impact of intervention on outcomes/utilization
Step 4: Disseminate results/Scale it up if it works
38. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
NEW YORK MEDICAID
CHRONIC ILLNESS DEMONSTRATION PROJECT
• NY Medicaid fee-for-service patients
• Adult disabled
• Not on Medicare (non-duals)
• Not in residential care
39. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
• Used five years of historic paid claims records
• Predicted hospitalization in next 12 months
• Ran quarterly
40. BASIC APPROACH
FOR NY MEDICAID CHRONIC ILLNESS
DEMONSTRATION PROJECT [CIDP]
Index
Quarters
Q1 Q2 Q3 Q4
Year 1 Year 2 Year 3 Year 4 Year 5
41. BASIC APPROACH
TYPES OF VARIABLES USED IN NY MEDICAID’S
CHRONIC ILLNESS DEMONSTRATION PROJECT [CIDP]
• Prior hospital utilization by type
– Number of admissions
– Intervals/recentness
• Prior emergency department utilization
• Prior outpatient utilization/claims
– By type of visit (primary care, specialty care, substance abuse, etc)
– By service type (transportation, home care, personal care, etc)
• Diagnostic information from prior hospital and outpatient utilization
– Chronic conditions (type/number)
– Hierarchical grouping (Hierarchical Condition Categories - HCCs)
• Prior costs
– Pharmacy
– DME
– Total
• Characteristics of the predominant hospital and primary care provider
• Patient characteristics: Age, gender, race/ethnicity, eligibility category
42. BASIC APPROACH
FOR NY MEDICAID CHRONIC ILLNESS
DEMONSTRATION PROJECT [CIDP]
Index
Quarters
Q1 Q2 Q3 Q4
Year 1 Year 2 Year 3 Year 4 Year 5
43. BASIC APPROACH
FOR NY MEDICAID CHRONIC ILLNESS
DEMONSTRATION PROJECT [CIDP]
Index Intervention
Quarters Quarters
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6
44. CASE FINDING ALGORITHM
NUMBER OF PATIENTS IDENTIFIED
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
50,000
45,000
40,000
Patients Identified
35,000
TOTAL FLAGGED
30,000 False
Positives
33%
25,000
20,000
15,000 False
Positives
15%
10,000 CORRECTLY FLAGGED False
Positives
5,000 7%
0
40 45 50 55 60 65 70 75 80 85 90 95
Risk Score Threshold
45. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
46. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
Demographic Characteristics
Risk Score Risk Score Risk Score All
> 50 > 75 > 90 Patients
N 33,363 8,713 2,176 64,446
Age 45.1 44.8 44.3 47.6
Female 43.9% 38.5% 34.7% 49.7%
NYC Fiscal County 72.2% 80.0% 84.4% 69.1%
White 28.2% 23.6% 22.9% 32.7%
Black 40.7% 48.1% 49.4% 33.1%
Hispanic 15.0% 14.2% 12.2% 14.6%
Other/Unknown 16.1% 14.2% 15.4% 19.5%
52. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
Selected Ambulatory Care Use Prior 12 Months
Risk Score Risk Score Risk Score All
> 50 > 75 > 90 Patients
Any primary care visit 71.7% 72.9% 68.3% 64.8%
Any speciatly care visit 39.2% 40.8% 39.9% 35.6%
No primary care visit 28.3% 27.1% 31.7% 35.2%
No PC/spec care visit 24.2% 22.6% 26.7% 31.3%
No PC/spec/OBGYN visit 23.7% 22.1% 26.1% 30.7%
Any psych visit 35.3% 35.8% 36.9% 29.6%
Any alcohol/drug visit 29.5% 38.8% 38.8% 19.5%
Any dental visit 37.3% 39.6% 37.5% 32.4%
Any home care 12.8% 17.2% 18.6% 8.5%
Any transportation 45.9% 61.1% 70.2% 32.2%
Any pharmacy 88.0% 89.5% 85.6% 78.3%
Any DME 18.7% 20.9% 20.5% 15.2%
Any comp case mgt 7.6% 10.8% 10.3% 5.2%
Any community rehab 1.1% 1.3% 0.8% 0.8%
53. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
Selected Ambulatory Care Use Prior 12 Months
Risk Score Risk Score Risk Score All
> 50 > 75 > 90 Patients
Any primary care visit 71.7% 72.9% 68.3% 64.8%
Any speciatly care visit 39.2% 40.8% 39.9% 35.6%
No primary care visit 28.3% 27.1% 31.7% 35.2%
No PC/spec care visit 24.2% 22.6% 26.7% 31.3%
No PC/spec/OBGYN visit 23.7% 22.1% 26.1% 30.7%
Any psych visit 35.3% 35.8% 36.9% 29.6%
Any alcohol/drug visit 29.5% 38.8% 38.8% 19.5%
Any dental visit 37.3% 39.6% 37.5% 32.4%
Any home care 12.8% 17.2% 18.6% 8.5%
Any transportation 45.9% 61.1% 70.2% 32.2%
Any pharmacy 88.0% 89.5% 85.6% 78.3%
Any DME 18.7% 20.9% 20.5% 15.2%
Any comp case mgt 7.6% 10.8% 10.3% 5.2%
Any community rehab 1.1% 1.3% 0.8% 0.8%
54. “MEDICAL HOME”
OUTPATIENT CARE
[PRIMARY/SPECIALTY/OB]
Looking back at two years of claims data, classify patients as:
• “Loyal” patients: 3+ visits with one provider having ≥ 50%
of visits during the 2-year period
• “Shoppers”: 3+ visits with no provider having ≥ 50%
of visits during the 2-year period
• “Occasional users”: Less than 3 visits during the 2-year
period
• “No PC/Spec/OB” patients: No primary care, specialty care,
or OB visits during the 2-year period
55. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
“Medical Home” for Patients with Risk Score ≥50
Based on Prior 2-Years of Ambulatory Use
Number of
All PC/Spec/OB
"Medical Home" Status
NYS Providers
Touched
Loyal 48.9% 2.80
OPD/Satellite 25.1% 2.97
D&TC 15.0% 2.55
MD 8.8% 2.71
Shopper 18.8% 5.39
Occasional User 13.3% 1.18
No PC/Spec/OB 19.0% 0.00
Total 100.0% 2.54
56. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
“Medical Home” for Patients with Risk Score ≥50
Based on Prior 2-Years of Ambulatory Use
Number of
All PC/Spec/OB
"Medical Home" Status
NYS Providers
Touched
Loyal 48.9% 2.80
OPD/Satellite 25.1% 2.97
D&TC 15.0% 2.55
MD 8.8% 2.71
Shopper 18.8% 5.39
Occasional User 13.3% 1.18
No PC/Spec/OB 19.0% 0.00
Total 100.0% 2.54
57. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
“Medical Home” for Patients with Risk Score ≥50
Based on Prior 2-Years of Ambulatory Use
Number of
All PC/Spec/OB
"Medical Home" Status
NYS Providers
Touched
Loyal 48.9% 2.80
OPD/Satellite 25.1% 2.97
D&TC 15.0% 2.55
MD 8.8% 2.71
Shopper 18.8% 5.39
51% Occasional User 13.3% 1.18
No PC/Spec/OB 19.0% 0.00
Total 100.0% 2.54
58. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
“Medical Home” for Patients with Risk Score ≥50
Based on Prior 2-Years of Ambulatory Use
Number of
All PC/Spec/OB
"Medical Home" Status
NYS Providers
Touched
Loyal 48.9% 2.80
OPD/Satellite 25.1% 2.97
D&TC 15.0% 2.55
MD 8.8% 2.71
Shopper 18.8% 5.39
Occasional User 13.3% 1.18
No PC/Spec/OB 19.0% 0.00
Total 100.0% 2.54
59. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
“Medical Home” for Patients with Risk Score ≥50
Based on Prior 2-Years of Ambulatory Use
Number of % of
Number of PC/Spec/OB Patients
All PC/Spec/OB Providers All
"Medical Home" Status
NYS Providers Touched NYS
Touched
1 Provider 0.0%
Loyal 48.9% 2.80 2 Providers 4.9%
OPD/Satellite 25.1% 2.97 3 Providers 22.7%
D&TC 15.0% 2.55 4-5 Providers 35.7%
MD 8.8% 2.71 5-9 Providers 28.8%
Shopper 18.8% 5.39 10+ Providers 8.0%
Occasional User 13.3% 1.18 Total 100.0%
No PC/Spec/OB 19.0% 0.00
Total 100.0% 2.54
60. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
Risk Score Risk Score Risk Score All
> 50 > 75 > 90 Patients
Costs Prior 12 Months
Inpatient 20,973 42,357 75,221 12,442
Emergency Department 306 576 1,040 199
Primary Care Visit 489 535 495 416
Specialty Care Visit 80 83 75 71
Psychiatric Care Visit 1,045 862 693 899
Substance Abuse Visit 1,129 1,342 1,070 748
Other Ambulatory 1,989 2,746 3,223 1,494
Pharmacy 6,470 7,711 7,545 4,905
Transportation 427 658 810 289
Community Rehab 109 112 57 73
Case Management 349 544 554 230
Personal Care 853 914 755 754
Home Care 875 1,201 1,357 601
LTHHC 49 116 214 29
All Other 2,388 3,500 3,738 1,738
Total Cost 37,530 63,259 96,848 24,885
Costs Next 12 Months
Inpatient 26,777 45,513 70,491 16,791
Emergency Department 299 527 921 198
Primary Care Visit 415 394 360 375
Specialty Care Visit 52 44 34 55
Psychiatric Care Visit 1,041 786 582 964
Substance Abuse Visit 1,155 1,320 1,061 796
Other Ambulatory 2,183 2,831 2,987 1,678
Pharmacy 7,246 7,726 7,194 5,834
Transportation 548 752 794 389
Community Rehab 170 184 59 173
Case Management 392 547 533 267
Personal Care 1,017 1,023 795 918
Home Care 1,229 1,327 1,392 986
LTHHC 117 117 63 110
All Other 3,895 5,071 5,409 3,089
Total Cost 46,537 68,162 92,674 32,622
61. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
Risk Score Risk Score Risk Score All
> 50 > 75 > 90 Patients
Costs Prior 12 Months
Inpatient 20,973 42,357 75,221 12,442
Emergency Department 306 576 1,040 199
Primary Care Visit 489 535 495 416
Specialty Care Visit 80 83 75 71
Psychiatric Care Visit 1,045 862 693 899
Substance Abuse Visit 1,129 1,342 1,070 748
Other Ambulatory 1,989 2,746 3,223 1,494
Pharmacy 6,470 7,711 7,545 4,905
Transportation 427 658 810 289
Community Rehab 109 112 57 73
Case Management 349 544 554 230
Personal Care 853 914 755 754
Home Care 875 1,201 1,357 601
LTHHC 49 116 214 29
All Other 2,388 3,500 3,738 1,738
Total Cost 37,530 63,259 96,848 24,885
Costs Next 12 Months
Inpatient 26,777 45,513 70,491 16,791
Emergency Department 299 527 921 198
Primary Care Visit 415 394 360 375
Specialty Care Visit 52 44 34 55
Psychiatric Care Visit 1,041 786 582 964
Substance Abuse Visit 1,155 1,320 1,061 796
Other Ambulatory 2,183 2,831 2,987 1,678
Pharmacy 7,246 7,726 7,194 5,834
Transportation 548 752 794 389
Community Rehab 170 184 59 173
Case Management 392 547 533 267
Personal Care 1,017 1,023 795 918
Home Care 1,229 1,327 1,392 986
LTHHC 117 117 63 110
All Other 3,895 5,071 5,409 3,089
Total Cost 46,537 68,162 92,674 32,622
66. CHARACTERISTICS OF PATIENTS FLAGGED
BY CASE FINDING ALGORITHM
CIDP ELIGIBLE - MODEL DEVELOPMENT RUN
Top 25 Principal Diagnosis of “Future Admissions”
ICD9 # of % Cumm
ICD9 Decription
Code Adms Total %
78039 CONVULSIONS NEC 538 0.7% 58.5%
29534 PARAN SCHIZO-CHR/EXACERB 509 0.6% 59.1%
5770 ACUTE PANCREATITIS 499 0.6% 59.7%
V5811 ANTINEOPLASTIC CHEMO ENC 493 0.6% 60.3%
30400 OPIOID DEPENDENCE-UNSPEC 438 0.5% 60.9%
30420 COCAINE DEPEND-UNSPEC 412 0.5% 61.4%
29680 BIPOLAR DISORDER NOS 402 0.5% 61.9%
25002 DMII WO CMP UNCNTRLD 392 0.5% 62.3%
30301 AC ALCOHOL INTOX-CONTIN 391 0.5% 62.8%
29634 REC DEPR PSYCH-PSYCHOTIC 385 0.5% 63.3%
40391 HYP KID NOS W CR KID V 362 0.4% 63.7%
5849 ACUTE RENAL FAILURE NOS 361 0.4% 64.2%
29690 EPISODIC MOOD DISORD NOS 358 0.4% 64.6%
5990 URIN TRACT INFECTION NOS 353 0.4% 65.0%
7802 SYNCOPE AND COLLAPSE 353 0.4% 65.5%
4660 ACUTE BRONCHITIS 337 0.4% 65.9%
30411 SED,HYP,ANXIOLYT DEP-CON 326 0.4% 66.3%
5589 NONINF GASTROENTERIT NEC 323 0.4% 66.7%
34590 EPILEP NOS W/O INTR EPIL 318 0.4% 67.0%
30480 COMB DRUG DEP NEC-UNSPEC 312 0.4% 67.4%
25013 DMI KETOACD UNCONTROLD 309 0.4% 67.8%
29532 PARANOID SCHIZO-CHRONIC 299 0.4% 68.2%
2910 DELIRIUM TREMENS 292 0.4% 68.5%
29633 RECUR DEPR PSYCH-SEVERE 282 0.3% 68.9%
25080 DMII OTH NT ST UNCNTRLD 281 0.3% 69.2%
67. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
68. CHARACTERISTICS OF
INTERVIEWED BELLEVUE PATIENTS
% of
Characteristic
Total
Marrital status
Married/living with partner 14%
Separated 16%
Divorced 10%
Widowed 4%
Never married 56%
Curently living alone 52%
No "close" frriends/relatives 16%
Two or fewer "close" friends/relatives 48%
Low "Perceived Availablity of Support" 42%
Bellevue Hospital Center
69. CHARACTERISTICS OF
INTERVIEWED BELLEVUE PATIENTS
% of
Characteristic
Total
Usual source of care
None 16%
58%
Emergency department 42%
OPD/Clinic 20%
Community based clinic 8%
Private/Group MD/other 14%
Bellevue Hospital Center
70. CHARACTERISTICS OF
INTERVIEWED BELLEVUE PATIENTS
% of
Characteristic
Total
Current housing status
Apartment/home rental 34%
Public housing 2%
Residential facility 2%
Staying with family/friends 24%
Shelter 60% 8%
Homeless 28%
Homeless anytime previous 2 years 50%
Bellevue Hospital Center
71. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
THEN GET SOME SMART PEOPLE IN THE ROOM
AND DESIGN THE INTERVENTION
73. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
74. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system
– Primary care
– Specialty care
– Substance abuse/mental health services
– Inpatient care
– Etc, etc
75. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
– Primary care
– Specialty care
– Substance abuse/mental health services
– Inpatient care
– Community based social support programs/resources
– Supportive housing for many
– Etc, etc, etc, etc
76. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
• Some sort of care/service-coordinator/arranger
– With a reasonable caseload size
– With a clear mission (to improve health and to reduce costs)
77. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
• Some sort of care/service-coordinator/arranger
• Core IT and analytic support capacity to…
– Track patient utilization in close to real time
– Mine administrative data and target interventions/outreach
– Provide analysis of utilization patterns
• Identify trends/problems to continuously re-design intervention strategies
• Provide feed-back to providers on performance
– Hospital admission rates
– ED visit rates
– Adherence to evidence based practice standards
– Support effective use of electronic medical records where available
78. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
• Some sort of care/service-coordinator/arranger
• Core IT and analytic support capacity
• Ability to provide real time support at critical junctures
– ED visit - prevention of “social admissions”
– Hospital discharge - effective community support/management planning
– Patient initiated - help for an emerging crisis
79. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
• Some sort of care/service-coordinator/arranger
• Core IT and analytic support capacity
• Ability to provide real time support at critical junctures
• Incentives/reimbursement policies to encourage and reward
“effective and cost efficient care”
– Hospitals must have a shared interest in avoiding admissions
– Reimbursement rates for OP services need to be related to their costs
– Costs of social support need to be recognized
– [No new money – any new/augmented services offset by IP savings]
80. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
• Some sort of care/service-coordinator/arranger
• Core IT and analytic support capacity
• Ability to provide real time support at critical junctures
• Incentives/reimbursement policies to encourage and reward
“effective and cost efficient care”
81. SO WHAT’S IT GOING TO TAKE?
• Multi-disciplinary approach for individualized needs
assessment and care planning for participating patients
• Integrated/organized/coordinated health care delivery
system and linkage to social service delivery system
• Some sort of care/service-coordinator/arranger
• Core IT and analytic support capacity ???
• Ability to provide real time support at critical junctures ???
• Incentives/reimbursement policies to encourage and reward
“effective and cost efficient care” ???
82. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
Step 3: Implement/evaluate pilot projects based on this
information
- Pilot with a quasi-experimental design (intervention/control)
- Conduct “formative” evaluation during early phases of
implementation
- Assess impact of intervention on outcomes/utilization
83. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
• NY State legislature authorized $20M demonstration
• After a competitive procurement process that took 13
months to implement, awards for 7 pilots March, 2009
• NYC sites had a goal of 500 patients, non-NYC 250
• Program provides $250/month for care coordination and
a “shared savings pool”
• July, 2009: One pilot dropped out
• August, 2009: Enrollment began in 6 remaining pilots
• October, 2009 – July, 2011: sites received quarterly
enrollment refreshments
84. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
• Federal government would not allow randomization of
patients into the initiative
– All patients in a geographic area must have access to same set of
services unless obtain a “waiver” (18 months and politically “fraught”)
85. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
• Federal government would not allow randomization of
patients into the initiative
– All patients in a geographic area must have access to same set of
services unless obtain a “waiver” (18 months and politically “fraught”)
– Brilliant solution: randomize zip codes (and tell federal government that
being implement only some areas of the state)
86. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
• Federal government would not allow randomization of
patients into the initiative
– All patients in a geographic area must have access to same set of
services unless obtain a “waiver” (18 months and politically “fraught”)
– Brilliant solution: randomize zip codes (and tell federal government that
being implement only some areas of the state)
• The $48 billion NY Medicaid agency (equivalent in revenue to
#57 on Fortune 500 list of U.S. companies) had no funds authorized
to conduct an evaluation
– Local philanthropy provided limited funding
– But not enough to survey patients or contact control group
87. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
• Federal government would not allow randomization of
patients into the initiative
– All patients in a geographic area must have access to same set of
services unless obtain a “waiver” (18 months and politically “fraught”)
– Brilliant solution: randomize zip codes (and tell federal government that
being implement only some areas of the state)
• The $48 billion NY Medicaid agency (equivalent in revenue to
#57 on Fortune 500 list of U.S. companies) had no funds authorized
to conduct an evaluation
– Local philanthropy provided limited funding
– But not enough to survey patients or contact control group
• Sites had enormous difficulty locating patients for enrollment
– Found and enrolled only 25% of eligible patients
– State dropped risk score cut-off from 50 to 40 and finally to 30
88. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
• Federal government would not allow randomization of
patients into the initiative
– All patients in a geographic area must have access to same set of
services unless obtain a “waiver” (18 months and politically “fraught”)
– Brilliant solution: randomize zip codes (and tell federal government that
being implement only some areas of the state)
• The $48 billion NY Medicaid agency (equivalent in revenue to
#57 on Fortune 500 list of U.S. companies) had no funds authorized
to conduct an evaluation
– Local philanthropy provided limited funding
– But not enough to survey patients or contact control group
• Sites had enormous difficulty locating patients for enrollment
– Found and enrolled only 25% of eligible patients
– State dropped risk score cut-off from 50 to 40 and finally to 30
89. HOW WE ALMOST GOT IT RIGHT,
BUT THEN NOT SO MUCH
Step 1: See if you can develop a predictive model to identify patients
with risks that you think you can do something about
Step 2: Learn as much as you can about these patients to help
in designing the intervention(s)
- Use available administrative data
- Apply predictive model to real patients – interview a sample of
these patients (and their providers, families, caregivers, etc.)
Step 3: Implement/evaluate pilot projects based on this
information
- Pilot with a quasi-experimental design (intervention/control)
- Conduct “formative” evaluation during early phases of
implementation
- Assess impact of intervention on outcomes/utilization
Step 4: Disseminate results/Scale it up if it works