State Health Access Reform Evaluation Lynn A. Blewett, Ph.D.  State Health Access Data Assistance Center AcademyHealth   S...
Overview of Presentation <ul><li>Quick Review-State Health Reform </li></ul><ul><li>Evaluation 101 </li></ul><ul><li>Examp...
Presentation Objectives <ul><li>To get you to think about measurable outcomes during your discussions of health reform </l...
Quick Overview of State Health Reform
Drivers of State Health Reform Activity <ul><li>Increasing number of uninsured </li></ul><ul><ul><li>Drop in employer-spon...
Continued Increase in Uninsured Source:  U.S. Census Bureau, Current Population Surveys (March), 1989-2006 New verificatio...
Drop in Employer-Sponsored Coverage (U.S.) Source: US. Census Bureau, Income, Poverty, and Health Insurance Coverage in th...
Increase in Uninsured Children, 2005-06 710,000 New Uninsured Children <200% FPL 200-399% FPL 400%+ FPL 220,000 340,000 15...
 
Increase in State Revenues Source:  Quarterly Summary of State and Local Government Tax Revenue, U. S Census Bureau. Annua...
Source: Congressional Budget Office Federal Relief?
States Are in Different Phases of  Reform Process Illinois  (kids) Maine Massachusetts PA  (kids) Vermont WI  (kids) Calif...
Framework for Reform  <ul><li>Public Sector  </li></ul><ul><li>Initiatives </li></ul>Employer-Based Initiatives Insurance ...
Health Reform Evaluation -101
Ideal Policy Making Process <ul><li>Informed Decision-Making Process </li></ul><ul><ul><li>Made based on best available da...
<ul><li>POLICY FORMULATION </li></ul><ul><li>Agenda Setting </li></ul><ul><li>Assessment of Options </li></ul><ul><li>Poli...
Performance Assessment/Evaluation <ul><li>OBJECTIVE  </li></ul><ul><ul><li>to  provide  information/data/evidence…  </li><...
 
Performance Assessment  or Program Evaluation? <ul><li>Performance Measurement </li></ul><ul><ul><li>ongoing assessment  <...
The Importance of Reform Evaluation <ul><li>Inform future state policy  </li></ul><ul><li>Justify program budget/financing...
Program Objectives <ul><li>Measurable results that the program sets out to achieve </li></ul><ul><li>SMART Objective </li>...
Levels of Measurement <ul><li>Population-based </li></ul><ul><ul><li>Data collected from sample of target population </li>...
Program Evaluation Data <ul><li>Quantitative Data </li></ul><ul><ul><li>Surveys, secondary data </li></ul></ul><ul><ul><li...
Performance Assessment <ul><li>How do you know if your health reform was successful? </li></ul><ul><li>Key Areas to Assess...
Examples: Performance Outcomes for Health Reform
Goals of Vermont Reform <ul><li>Increase Access to Affordable Health Care Coverage </li></ul><ul><ul><li>More than 96% of ...
Goals of Maine Reform <ul><li>Increase Access to Affordable Health Care Coverage </li></ul><ul><ul><li>Aim to insure up to...
Additional Examples <ul><li>Process Measurement </li></ul><ul><ul><li>Time from first contact to enrollment </li></ul></ul...
Key Dimensions of Reform <ul><li>Increase  access and coverage  plus demonstrating that reform initiative is: </li></ul><u...
Data Sources – State vs. National
Assumption “ Good policy needs good data  and collecting good data is good policy.” Source:  Ernie Ingles, Chief Librarian...
Current Population Survey (CPS) <ul><li>Annual estimate of distribution of health insurance coverage </li></ul><ul><li>Rel...
CPS Also Key to SCHIP Funding <ul><li>Formula  inputs: estimates of low-income children and low-income uninsured children ...
SCHIP Evaluation Requirements  <ul><li>States required to monitor changes in levels of uninsurance for low-income kids </l...
HRSA-SPG Funded Survey Activity  2000-2005 Household and Employer Survey Employer Survey Household Survey
Why States Conduct Surveys <ul><li>Typically more sample than CPS </li></ul><ul><li>Ability to drill down to subpopulation...
On Average Uninsurance Estimates from State Surveys are 22% Lower than CPS   Uninsurance Estimates for Select States
Increased Sample Size in State Surveys
States Use of Data <ul><li>Four primary areas of policy development: </li></ul><ul><li>(1) To develop reliable and accurat...
<ul><ul><li>Focus Groups </li></ul></ul><ul><ul><li>(3) To collect information about opinions,    attitudes, and values </...
Technical Assistance  -  State Health Access Reform Evaluation  - State Health Access Data Assistance Center
State Health Access Data Assistance Center <ul><li>Funded by the Robert Wood Johnson Foundation </li></ul><ul><ul><li>Univ...
State Health Access  Reform Evaluation <ul><li>National Program Office of the RWJF  </li></ul><ul><ul><li>Co-located with ...
State Health Reform If States are the Laboratories….. Where are the lab reports?
Concluding Comments  <ul><li>Staff - Ask key legislative leaders what they will want to know next year? </li></ul><ul><li>...
More Concluding Comments <ul><li>Don’t forget to think about….  </li></ul><ul><li>Performance assessment  and </li></ul><u...
The SHADAC Vision Federal  Data (e.g., CPS) Better Understanding of the Characteristics of the Uninsured State Data (e.g.,...
Contact Information www.shadac.org www.statereformevaluation.org University of Minnesota School of Public Health Division ...
Upcoming SlideShare
Loading in...5
×

State Health Access Reform Evaluation

611

Published on

Presentation by Lynn Blewett at the AcademyHealth State Coverage Initiatives (SCI) State Coverage Institute, September 26 2007.

Published in: Health & Medicine, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
611
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • State Health Access Reform Evaluation

    1. 1. State Health Access Reform Evaluation Lynn A. Blewett, Ph.D. State Health Access Data Assistance Center AcademyHealth State Coverage Initiatives (SCI) State Coverage Institute Chicago, Illinois September 26, 2007 Funded by a grant from the Robert Wood Johnson Foundation
    2. 2. Overview of Presentation <ul><li>Quick Review-State Health Reform </li></ul><ul><li>Evaluation 101 </li></ul><ul><li>Examples: Performance Outcomes for Health Reform </li></ul><ul><li>Data sources – State vs. National </li></ul><ul><li>Technical Assistance </li></ul><ul><ul><li>State Health Access Reform Evaluation </li></ul></ul><ul><ul><li>State Health Access Data Assistance Center </li></ul></ul>
    3. 3. Presentation Objectives <ul><li>To get you to think about measurable outcomes during your discussions of health reform </li></ul><ul><li>To be specific about your expectations for reform initiatives </li></ul><ul><ul><li>How you will know if the program achieved its objectives? What data will you need? </li></ul></ul><ul><li>To talk to each other and reach consensus on expectations….. </li></ul><ul><li>Think about state-specific data needs and build those into your legislation </li></ul>
    4. 4. Quick Overview of State Health Reform
    5. 5. Drivers of State Health Reform Activity <ul><li>Increasing number of uninsured </li></ul><ul><ul><li>Drop in employer-sponsored coverage </li></ul></ul><ul><ul><li>Kids impact moderated by SCHIP </li></ul></ul><ul><ul><li>No safety net for adults </li></ul></ul><ul><li>Increasing number of underinsured </li></ul><ul><ul><li>Higher out-of-pocket costs </li></ul></ul><ul><li>Relatively positive state revenue climate </li></ul><ul><li>Lack of national efforts for reform </li></ul><ul><ul><li>Iraq, immigration, etc. dominating Congress </li></ul></ul>
    6. 6. Continued Increase in Uninsured Source: U.S. Census Bureau, Current Population Surveys (March), 1989-2006 New verification question Millions of Uninsured, all ages 15.8% of Population
    7. 7. Drop in Employer-Sponsored Coverage (U.S.) Source: US. Census Bureau, Income, Poverty, and Health Insurance Coverage in the United States: 2006.
    8. 8. Increase in Uninsured Children, 2005-06 710,000 New Uninsured Children <200% FPL 200-399% FPL 400%+ FPL 220,000 340,000 150,000 31.3% 47.5% 21.2% Note: 200% to 399% of the federal poverty level (FPL) is roughly $40,000-$80,000 in annual income for a family of four in 2006. Source: KCMU/Urban Institute analysis of the March 2007 CPS .
    9. 10. Increase in State Revenues Source: Quarterly Summary of State and Local Government Tax Revenue, U. S Census Bureau. Annual Change in 2nd Quarter State and Local Tax Revenue Drop in 2007?
    10. 11. Source: Congressional Budget Office Federal Relief?
    11. 12. States Are in Different Phases of Reform Process Illinois (kids) Maine Massachusetts PA (kids) Vermont WI (kids) California Oregon WI Colorado Minnesota Illinois (all ) New Jersey New Mexico PA (all) WI (all) Post –Implementation Legislation Passed/ Implementation Legislation Introduced In Discussion
    12. 13. Framework for Reform <ul><li>Public Sector </li></ul><ul><li>Initiatives </li></ul>Employer-Based Initiatives Insurance Reform Tax credits HIFA expansion waivers SCHIP premium assistance Small employer purchasing pools
    13. 14. Health Reform Evaluation -101
    14. 15. Ideal Policy Making Process <ul><li>Informed Decision-Making Process </li></ul><ul><ul><li>Made based on best available data and analysis </li></ul></ul><ul><ul><li>Decision maker is knowledgeable, interested and has best information and advice </li></ul></ul><ul><ul><li>Science and methods are sound </li></ul></ul><ul><ul><li>Thoughtful discussion of alternatives </li></ul></ul><ul><ul><li>Public is informed </li></ul></ul>
    15. 16. <ul><li>POLICY FORMULATION </li></ul><ul><li>Agenda Setting </li></ul><ul><li>Assessment of Options </li></ul><ul><li>Political Setting </li></ul>POLICY IMPLEMENTATION POLICY Adoption POLICY MODIFICATION Program Evaluation , Assess impact of policies Influence future policy formulation INPUTS Preferences of individuals, organizations, interest groups , along with cultural Demographics, ethical, legal, social and technological inputs Rulemaking Operation Legislation Window of Opportunity Source: Longest, 1998 Longest, 1988.Policy Making Process in the US
    16. 17. Performance Assessment/Evaluation <ul><li>OBJECTIVE </li></ul><ul><ul><li>to provide information/data/evidence… </li></ul></ul><ul><ul><li>to support resource allocation and other policy decisions…. </li></ul></ul><ul><ul><li>to improve service delivery and program effectiveness. </li></ul></ul>
    17. 19. Performance Assessment or Program Evaluation? <ul><li>Performance Measurement </li></ul><ul><ul><li>ongoing assessment </li></ul></ul><ul><ul><li>use of performance standards </li></ul></ul><ul><ul><li>early warning system to management </li></ul></ul><ul><ul><li>vehicle for improving accountability to the public </li></ul></ul><ul><li>Program Evaluation </li></ul><ul><ul><li>typically more in-depth examination of program </li></ul></ul><ul><ul><li>broader context of environment and alternatives </li></ul></ul><ul><ul><li>overall assessment of whether the program works </li></ul></ul><ul><ul><li>identification of adjustments that may improve its results </li></ul></ul>Source: U.S. Government Accountability Office, 1998.
    18. 20. The Importance of Reform Evaluation <ul><li>Inform future state policy </li></ul><ul><li>Justify program budget/financing </li></ul><ul><li>Satisfy legislative requirements </li></ul><ul><li>Identify successful initiatives </li></ul><ul><li>Change course if program not meeting benchmarks </li></ul><ul><li>Inform state and national debate on health reform </li></ul><ul><ul><li>What works at the state level? </li></ul></ul>Source: CMS Evaluating Demonstrations: A technical assistance guide for states.
    19. 21. Program Objectives <ul><li>Measurable results that the program sets out to achieve </li></ul><ul><li>SMART Objective </li></ul><ul><ul><li>S – Specific </li></ul></ul><ul><ul><li>M – Measurable </li></ul></ul><ul><ul><li>A – Appropriate </li></ul></ul><ul><ul><li>R – Realistic </li></ul></ul><ul><ul><li>T – Time bound </li></ul></ul><ul><li>Focus of objective </li></ul><ul><ul><li>What will change? </li></ul></ul><ul><ul><li>Not what the program will do </li></ul></ul>
    20. 22. Levels of Measurement <ul><li>Population-based </li></ul><ul><ul><li>Data collected from sample of target population </li></ul></ul><ul><ul><li>Survey data, focus groups </li></ul></ul><ul><li>Program-based </li></ul><ul><ul><li>Data collected from program recipients, clients, or participants </li></ul></ul><ul><ul><li>Administrative data </li></ul></ul>
    21. 23. Program Evaluation Data <ul><li>Quantitative Data </li></ul><ul><ul><li>Surveys, secondary data </li></ul></ul><ul><ul><li>Measures activities carried out </li></ul></ul><ul><ul><li>Use to assess extent of program utilization, behavior change, or cost per unit of change </li></ul></ul><ul><li>Qualitative Data </li></ul><ul><ul><li>Interviews, focus groups, observations, document review </li></ul></ul><ul><ul><li>Use to understand attitudes, beliefs, reactions, to assess quality, to explain perceptions </li></ul></ul>
    22. 24. Performance Assessment <ul><li>How do you know if your health reform was successful? </li></ul><ul><li>Key Areas to Assess: </li></ul><ul><ul><li>Coverage </li></ul></ul><ul><ul><li>Cost and Efficiency </li></ul></ul><ul><ul><li>Fairness and Equity </li></ul></ul><ul><ul><li>Choice and Autonomy </li></ul></ul><ul><li>Develop goals that are measurable, assessable and realistic. </li></ul><ul><li>Consider short-term and long-term outcomes. </li></ul>
    23. 25. Examples: Performance Outcomes for Health Reform
    24. 26. Goals of Vermont Reform <ul><li>Increase Access to Affordable Health Care Coverage </li></ul><ul><ul><li>More than 96% of Vermonters will have coverage. </li></ul></ul><ul><li>Improve Quality of Care </li></ul><ul><ul><li>100% of Medicaid, Dr. Dynasaur, and VHAP beneficiaries living with a chronic condition will receive health care services that are based on nationally recognized clinical best-practice guidelines for health treatments and self care. </li></ul></ul><ul><li>Contain Costs </li></ul><ul><ul><li>The cost shift resulting from insufficient Medicaid reimbursement rates will be reduced. </li></ul></ul>Source: Vermont Health Care Reform: 5 Year Strategic Plan
    25. 27. Goals of Maine Reform <ul><li>Increase Access to Affordable Health Care Coverage </li></ul><ul><ul><li>Aim to insure up to 31,000 individuals during its first year and provide access to coverage for at least another estimated 110,000 individuals by 2009. </li></ul></ul><ul><li>Improve Quality of Care </li></ul><ul><li>Contain Costs through Voluntary Caps </li></ul><ul><ul><li>Dirigo Health requests hospitals and other providers to limit their cost growth to 3% per year and their operating margins to 3.5%. Insurers are asked to limit their operating margin to 3.5%. </li></ul></ul>http://www.dirigohealth.maine.gov/dhlp01.html
    26. 28. Additional Examples <ul><li>Process Measurement </li></ul><ul><ul><li>Time from first contact to enrollment </li></ul></ul><ul><ul><li>Waiting list, backup list </li></ul></ul><ul><ul><li># of contacts per outreach effort </li></ul></ul><ul><ul><li># of contacts that result in enrollment </li></ul></ul><ul><li>Outcomes Measurement </li></ul><ul><ul><li>Increase coverage for low-income children by 30% </li></ul></ul><ul><ul><li>Increase the number of plans offering insurance </li></ul></ul><ul><ul><li>Reduce growth in public program spending by 10% </li></ul></ul><ul><ul><li>Increase immunization rates for non-white populations to meet 2010 goals </li></ul></ul><ul><ul><li>Estimates of direct substitution of public for employer-based plan (i.e., Crowd Out) </li></ul></ul>
    27. 29. Key Dimensions of Reform <ul><li>Increase access and coverage plus demonstrating that reform initiative is: </li></ul><ul><li>Affordable </li></ul><ul><li>Sustainable </li></ul><ul><li>Efficient </li></ul>
    28. 30. Data Sources – State vs. National
    29. 31. Assumption “ Good policy needs good data and collecting good data is good policy.” Source: Ernie Ingles, Chief Librarian, University of Alberta
    30. 32. Current Population Survey (CPS) <ul><li>Annual estimate of distribution of health insurance coverage </li></ul><ul><li>Released by the Census Bureau every August </li></ul><ul><li>Only federal survey with uninsurance estimates for all 50 states and DC </li></ul><ul><li>Used to compare coverage and change in coverage across states and for nation </li></ul>
    31. 33. CPS Also Key to SCHIP Funding <ul><li>Formula inputs: estimates of low-income children and low-income uninsured children in each state </li></ul><ul><ul><li>Pooled years of CPS data </li></ul></ul><ul><ul><li>Still small sample size, especially for small states </li></ul></ul><ul><ul><li>Bias in data skews distribution of funds </li></ul></ul><ul><li>State administrative data is not used in SCHIP funding formula </li></ul><ul><ul><li>More kids enrolled, less uninsured, less allotment </li></ul></ul>Is there a better state-level data source to inform formula inputs?
    32. 34. SCHIP Evaluation Requirements <ul><li>States required to monitor changes in levels of uninsurance for low-income kids </li></ul><ul><ul><li>Back to CPS? </li></ul></ul><ul><ul><li>State survey? </li></ul></ul><ul><ul><li>Some state using administrative counts to show increase in public program enrollment </li></ul></ul><ul><li>New- D emonstrate that public program expansion does not lead to drop in employer sponsored coverage </li></ul><ul><ul><li>Enrollee surveys? </li></ul></ul><ul><ul><li>Employer surveys? </li></ul></ul>
    33. 35. HRSA-SPG Funded Survey Activity 2000-2005 Household and Employer Survey Employer Survey Household Survey
    34. 36. Why States Conduct Surveys <ul><li>Typically more sample than CPS </li></ul><ul><li>Ability to drill down to subpopulations </li></ul><ul><ul><li>Children </li></ul></ul><ul><ul><li>Geographic Units </li></ul></ul><ul><ul><li>Race/ethnicity </li></ul></ul><ul><li>Analysts have data in hand </li></ul><ul><ul><li>Better knowledge of data </li></ul></ul><ul><ul><li>Ability to do analysis in-house </li></ul></ul>
    35. 37. On Average Uninsurance Estimates from State Surveys are 22% Lower than CPS Uninsurance Estimates for Select States
    36. 38. Increased Sample Size in State Surveys
    37. 39. States Use of Data <ul><li>Four primary areas of policy development: </li></ul><ul><li>(1) To develop reliable and accurate estimates of the number of uninsured </li></ul><ul><ul><li>To facilitate state-specific policy discussions </li></ul></ul><ul><li>(2) To identify the demographic, economic and health-related characteristics of the uninsured </li></ul><ul><ul><li>To help target programs and outreach </li></ul></ul>
    38. 40. <ul><ul><li>Focus Groups </li></ul></ul><ul><ul><li>(3) To collect information about opinions, attitudes, and values </li></ul></ul><ul><ul><li>-- To facilitate the design policy options </li></ul></ul><ul><ul><li>-- To assess political feasibility </li></ul></ul><ul><ul><li>(4) To understand the motivations of uninsured populations and employers </li></ul></ul><ul><ul><li>-- To design approach and incentives </li></ul></ul>States Use of Data (2)
    39. 41. Technical Assistance - State Health Access Reform Evaluation - State Health Access Data Assistance Center
    40. 42. State Health Access Data Assistance Center <ul><li>Funded by the Robert Wood Johnson Foundation </li></ul><ul><ul><li>University of Minnesota, School of Public Health </li></ul></ul><ul><li>Goals: </li></ul><ul><ul><li>Help states monitor rates of health insurance coverage using state and federal data </li></ul></ul><ul><ul><li>Research factors associated with access and coverage </li></ul></ul><ul><ul><li>Provide targeted policy analysis and technical assistance to states </li></ul></ul><ul><li>Bridge between state and federal agencies; between survey data and state health policy </li></ul><ul><li>Technical assistance on use of federal survey data and best practices on state surveys </li></ul>
    41. 43. State Health Access Reform Evaluation <ul><li>National Program Office of the RWJF </li></ul><ul><ul><li>Co-located with SHADAC at University of MN, SPH </li></ul></ul><ul><li>Goals: </li></ul><ul><ul><li>Fund and coordinate evaluations of state health reform </li></ul></ul><ul><ul><li>Identify and fill gaps on research to identify what works and why </li></ul></ul><ul><ul><li>Organize and disseminate findings in a manner that is meaningful and user-friendly </li></ul></ul><ul><ul><li>Inform state and national policy on health care access and coverage </li></ul></ul><ul><li>Technical assistance on program evaluation and assessment </li></ul>
    42. 44. State Health Reform If States are the Laboratories….. Where are the lab reports?
    43. 45. Concluding Comments <ul><li>Staff - Ask key legislative leaders what they will want to know next year? </li></ul><ul><li>Discuss long- and short-term program objectives and identify the monitoring process up front </li></ul><ul><li>Make sure staff understand legislative/policy objectives </li></ul><ul><li>Make sure policy makers understand limitations of data/measurement </li></ul><ul><li>Build performance assessment into legislative initiatives </li></ul>
    44. 46. More Concluding Comments <ul><li>Don’t forget to think about…. </li></ul><ul><li>Performance assessment and </li></ul><ul><ul><li>Monitoring along the way </li></ul></ul><ul><ul><li>Likely within state government </li></ul></ul><ul><li>Program evaluation </li></ul><ul><ul><li>How are we going to know in 2-3 years if the program achieved its objectives? </li></ul></ul><ul><ul><li>Possible outside vendor/contractor </li></ul></ul><ul><li>Think now about what you will want to know later </li></ul><ul><ul><li>It’s more difficult to go back and evaluate a program without good baseline data and information </li></ul></ul>
    45. 47. The SHADAC Vision Federal Data (e.g., CPS) Better Understanding of the Characteristics of the Uninsured State Data (e.g., State Surveys) Increase Coverage and Access THANK YOU!
    46. 48. Contact Information www.shadac.org www.statereformevaluation.org University of Minnesota School of Public Health Division of Health Policy and Management 2221 University Avenue, Suite 345 Minneapolis Minnesota 55414 (612) 624-4802 Principal Investigator: Lynn A. Blewett, Ph.D. (blewe001@umn.edu)

    ×