Data and Methodologies Used in
the
Evaluation of Health Reform at
the State Level
Donna Spencer, PhD
State Health Access D...
Agenda
• State Health Access Reform Evaluation
(SHARE) Grant Program
• Systematic review of grant methods and data
• Data ...
About SHARE
• National Program of the Robert Wood Johnson Foundation
(RWJF) since 2006
• Goals:
– support the evaluation o...
About SHARE
• 3 rounds of grant awards
– 33 grants funded to date
– 9 currently in the field

• Over $7 million in researc...
States Studied by SHARE Grants

Studied by one or more
single- or multi-state
share grant(s)

Map excludes all-state/natio...
Policies and Programs Studied

Grants may be assigned to more than one policy and/or program.
6
Topics and Outcomes Studied

Grants may be assigned to more than one topic and/or outcome.
7
Systematic Review of Grants: Approach
• Excel-based abstraction tool
• Data abstracted:
–
–
–
–
–

Type of study/evaluatio...
Types of Data Used by Grantees*

* Grants may be assigned to more than one data type.
** Administrative data includes elig...
Federal and State Surveys Used

10
Survey Data Lessons
Federal Surveys

State Surveys

•

•

•
•
•

Existing data
– Time and $ resource
efficient
Some have g...
Medical Claims Data Used

12
Claims Data Lessons
• Ideal for measuring health care utilization and costs
• Does not rely on patient recall of health ca...
Administrative Data Used

14
Administrative Data Lessons
• Key outcomes of interest include enrollment, insurance
take-up, continuity in coverage, chur...
Qualitative Data Lessons
• Ideal for assessing
– Political/social/historical context of the program/policy
– Perspectives ...
Conclusions
•
•
•

•

•

Wealth of data available for health reform evaluation research
No one data source has it all
Shif...
Contact Information
Donna Spencer, PhD
SHARE Deputy Director
dspencer@umn.edu

©2002-2009 Regents of the University of Min...
Upcoming SlideShare
Loading in …5
×

Aea presentation final 18_oct2013

278 views
220 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
278
On SlideShare
0
From Embeds
0
Number of Embeds
89
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Aea presentation final 18_oct2013

  1. 1. Data and Methodologies Used in the Evaluation of Health Reform at the State Level Donna Spencer, PhD State Health Access Data Assistance Center University of Minnesota, Twin Cities AEA: Evaluation 2013 Washington, DC October 18, 2013 Supported by a grant from The Robert Wood Johnson Foundation
  2. 2. Agenda • State Health Access Reform Evaluation (SHARE) Grant Program • Systematic review of grant methods and data • Data sources used in health reform research and evaluation – – – – – Federal surveys State surveys Administrative data Medical claims data Qualitative methods • Lessons learned from the SHARE program 2
  3. 3. About SHARE • National Program of the Robert Wood Johnson Foundation (RWJF) since 2006 • Goals: – support the evaluation of health policy reform at the state level – develop an evidence-based resource to inform health reform efforts in the future • Focus: State-level reform and state implementation of national reform • Operated out of the State Health Access Data Assistance Center (SHADAC) in the Division of Health Policy and Management, School of Public Health, University of Minnesota. • Collaborators on this presentation: – Kelsey Avery (Graduate Research Assistant) – Carrie Au-Yueng, MPH (Research Fellow) – Lynn Blewett, PhD (SHADAC and SHARE Principal Investigator) 3
  4. 4. About SHARE • 3 rounds of grant awards – 33 grants funded to date – 9 currently in the field • Over $7 million in research and evaluation funding to date • Projects have ranged from 3-30 months in duration • Grantee institutions: mostly universities but also private research organizations and state agencies • States studied: – Single-state (14 grants) – Multi-state (10 grants) – All states/national (9 grants) 4
  5. 5. States Studied by SHARE Grants Studied by one or more single- or multi-state share grant(s) Map excludes all-state/national studies. 5
  6. 6. Policies and Programs Studied Grants may be assigned to more than one policy and/or program. 6
  7. 7. Topics and Outcomes Studied Grants may be assigned to more than one topic and/or outcome. 7
  8. 8. Systematic Review of Grants: Approach • Excel-based abstraction tool • Data abstracted: – – – – – Type of study/evaluation Quantitative/qualitative methods Types of data used and data sources Facilitators/obstacles in research/evaluation Methodological lessons learned • Grant documents used in review: – Proposals – Grant progress reports – Grant deliverables (presentations, publications, substantive reports, briefs) 8
  9. 9. Types of Data Used by Grantees* * Grants may be assigned to more than one data type. ** Administrative data includes eligibility data and enrollment data. 9
  10. 10. Federal and State Surveys Used 10
  11. 11. Survey Data Lessons Federal Surveys State Surveys • • • • • Existing data – Time and $ resource efficient Some have good state sample sizes Can facilitate state comparisons No one survey offers it all in terms of policy-relevant content and ample state data • • • • Larger state-specific sample sizes (in some cases) Targeted oversampling Questionnaire more easily modified and relevant for local policy environment Inconsistency/uncertainty in funding Own methodological limitations 11
  12. 12. Medical Claims Data Used 12
  13. 13. Claims Data Lessons • Ideal for measuring health care utilization and costs • Does not rely on patient recall of health care services • But precludes care paid for by a different payer or not paid for by health plan • Lacks good socio-demographic data (unlike surveys) • Large patient populations but comparison groups may be limited • Access to data can be difficult – Authorizations, data use agreements, competing demands – APCDs not viable choice in some states • Time consuming and more complicated to obtain, prepare, and analyze • Relationship with source agency essential 13
  14. 14. Administrative Data Used 14
  15. 15. Administrative Data Lessons • Key outcomes of interest include enrollment, insurance take-up, continuity in coverage, churning • Large patient populations • As with claims, not designed for research purposes per se – Data elements important to research may be limited • As with claims, access may be difficult • As with claims, time consuming and more complicated to obtain, prepare, and analyze • Relationship with source agency essential 15
  16. 16. Qualitative Data Lessons • Ideal for assessing – Political/social/historical context of the program/policy – Perspectives related to processes, implementation, outcomes • State staff/officials and other stakeholders motivated to participate • Constraints – Both national and state health reform have state agencies maxed out! – Other typical competing demands: legislative sessions, recent political developments, regular program schedules – Turnover in state program personnel 16
  17. 17. Conclusions • • • • • Wealth of data available for health reform evaluation research No one data source has it all Shifts occurring among relevant data sources – APCDs – National health reform has triggered new data needs and existing federal and state data sources are responding – New potential data sources (e.g., marketplaces) Relationship with state program important in state health reform evaluation for a host of reasons, including data access – Allocating funds for their role as well as data acquisition and preparation – Evaluation timelines need to accommodate IRB reviews may require extra time and attention especially with administrative/claims data sources 17
  18. 18. Contact Information Donna Spencer, PhD SHARE Deputy Director dspencer@umn.edu ©2002-2009 Regents of the University of Minnesota. All rights reserved. The University of Minnesota is an Equal Opportunity Employer 18

×