Framework can be A broad framework helps focus attention on:Big picture goalsHow each component (e.g., Medicaid, exchange) contributes to those goalsOutcomes, not just processesWhere it makes sense, ensure that individual programs are collecting data and measuring consistentlySelect a lead agency or person to be accountable
Scope: Selected your domainsFocus: articulated your policy goalsMeasure: determined the mechanisms for achieving policy goals, keeping in mind near/medium/long termLevel of measurement: Given some thought to the level of measurement – system/population, subpopulation geography, employer size, health provider tpe
overall enrollment, completed applications, transfers from CHIP to Medicaid, spells of enrollment, retention rates, new applicants versus re-entries, reasons for denials, and reasons for disenrollment
Churn: number of program disenrollees in a given month who later reenroll in the program following a gap in coverage of one to six months. (Chris Trenholm)Coverage from admin data – what time frame? Who is excluded (elderly)? Point in time?**Don’t need to remake wheel, use exiting definitions (other data frameworks), SHADAC can helpCoverage: uninsured? Public program – all or aggregate?System/populationSpecific population groups – e.g., age, income, health insurance coverage type, race/ethnicity, immigration statusGeographyEmployer sizeHealth care provider type – e.g., safety net providers
Start with list of priority measures, then do data scanOtherwise:might miss key measures of interest highlight less important measure just because you have the data.Do a lot of work assessing data sources on topics that don’t make the cutSurvey dataAdministrative dataData from health carriers, hospitals, providersOther?Level of Geography Subpopulation analysis , Available benchmarks, Timeliness, Ability to trend, Breadth and depth of topics Methodology, Ease of use and procurementConsider ways of collecting additional data through existing collection efforts : Existing state surveysProvider licensure process State tax returnIdentify data that might come out of new systems/processes Enlist outside support (e.g., state foundations)
51 measuresVery focused on IDing data gapsGoalsInform stakeholders Help CHCF prioritize next steps and resources for filling data gapsBuild coalitions and momentum to move process forwardApproach6 structured group discussions over 3 daysProfessional facilitatorRange of invited participants: advocates, providers, safety net, legislative staff, state and county government, insurers, researchers, foundationsTried to keep groups of “like minded” together
Coverage, continued from previous slide:Health insurance exchange: The insurance exchange plays a key role in the ACA’s coverage reforms, both as a vehicle for subsidies and as a means of organizing the market and making it easier for individuals and employers to shop for coverage. To understand how well the exchanges are performing these functions, trends in enrollment should be tracked over time (both in total and as a percentage of the individual and relevant employer markets, and separately for subsidized vs non-subsidized coverage).AAt a high level, the major gaps in data related to health insurance coverage relate to compliance with the Affordable Care Act’s individual mandate and employer requirements, and to the health insurance exchange. These are the sort of gaps that represent fundamental changes that are specific to the ACA and that can’t be filled until 2014. However, it’s important to be planning now for how to collect this information in a way that will be reliable and useful. REFERENCE TO LYNN’S PRESENTAITON ON REPORTING REQUIREMENTS UNDER THE ACA?One thing to note is that gaps for these recommended measures will of course look very different in differnet states. CA has a pretty rich existing data infrastructure to draw from, and this won’t be the case in all states. For example, we recommend the CHIS and CEHBS in CA for many measures, but in states that don’t conduct their own state specific population and employer based surveys other alternatives, such as the NHIS and MEPS-IC would have to be considered, and in some cases, such as the NHIS in smaller states, data sources may not be available at all.
With regard to affordability and comprehensiveness, there are significant gaps in the information available now, especially with regard to the nongroup health insurance market. We know very little right now about premiums, enrollment, and what kinds of products are being purchased in this market. We know more about the market for employer-sponsored coverage, but will need to track comprehensiveness of coverage in new ways once the exchanges are implemented. And finally, it will also be important to track premium and cost sharing subsidies that are provided through the exchanges beginning in 2014.
With regard to individuals’ access to care, the main gap related to use of services relates to the percentage of people who have preventive care visits; but there are several important gaps related to barriers to care that individuals experience. Specifically, these relate to people being able to get appointments in a timely way, having difficulty finding a provider that will accept new patients, and having difficulty finding a provider that accepts their insurance; it’s probably a good idea to track these measures separately for primary care and specialty care. Key gaps in the information that will be important to track access to care at a system level include the percentage of physicians who are accepting new patients, and the percentage that participate in Medi-Cal and Healthy Families. Again, tracking these measures separately for primary care and specialty care will be useful. And finally, the gaps in information related to the safety net relate mainly to the fact that the safety net system is pretty fragmented, with no data sources that provide a picture of the system as a whole. Many of the California data experts that we talked to mentioned the lack of information from county clinics as a very important gap here, since these clinics play a crucial role in delivering safety net services. There are also gaps in available information about the volume and cost of indigent care provided by counties. And finally, there are also some gaps in the data for hospitals – because not all hospitals report their financial information the same way, the picture of uncompensated care at the state level is currently incomplete.Transition to Barbara
25 measure's in totalState staff drivenDomains were generated based on the state goals of the exchange
Health Equity Number of individuals that attempt to obtain coverage through exchange (Access)Distribution of insurance status (Access)Number of individuals receiving premium subsidies (Affordability)Number of individuals receiving cost sharing subsidies (Affordability)Percentage of adults who cannot afford a doctor visit (Affordability)Percent of families with high cost burden (Affordability)Composite measure of satisfaction (Consumer Satisfaction)
Funded by a grant from the Robert Wood Johnson FoundationThinking Ahead – Monitoring the Impactof Health ReformElizabeth Lukanen, MPHSHADAC, University of MinnesotaNational Association of Medicaid Directors (NAMD)Spring MeetingDenver, ColoradoMay 21, 2013
Presentation Overview• Why should states develop a monitoringframework (and why should Medicaid beinvolved)?• Steps to develop a monitoring or evaluationframework• State examples2
2015 and Beyond4Everyone will be clamoring for dataand analysis on the impact of healthreform• States will be looking to report on “earlywins”• Policymakers and operational staff willneed information to make ongoingimplementation decisions• Heated debate is likely to continue and both sides will belooking for information on the impact• The media will be looking for ANY story• The public and key stakeholders will want a progressreport
Objectives for Generating aMonitoring/Evaluation Framework• Encourages agreement on goals,priorities, and how progress will bemeasured• Defines how each component ofreform (e.g., Medicaid, exchange)contributes to those goals• Establishes program/agencycollaboration to focus on the “bigpicture”• Avoids duplication of data collectionand provides consistency inmeasurement5• Provides opportunity to select lead agency or individual accountable formonitoring efforts• Prepares state staff to respond to future questions from policymakers
6o National surveys and analyses are a great, especiallywhen cross-state comparisons are important, but…• Each state will be unique in how it implements the ACA• State-led efforts will track progress toward statepriorities• States often have richer data to examine questions in-depthWhy Should Monitoring Efforts be State-Led?Why not just rely onnational studies or 50-state analyses fromother sources?
Why Should Medicaid Play a Role?7• Medicaid is “where it’s at”– Even if you don’t plan for it, you will likely engage inevaluation/monitoring work• Many key evaluation measures will rely onMedicaid data– Assure consistency in reporting– Avoid duplication of data collection and analysis– Reduce analyst burden• Define what it means to be successful• Contribute to and be aware of themessaging regarding impact of reform
Why Now?• Define in advance what is important to measure –helps identify successes and problem areas• Establish a baseline prior to reform implementation• Identify gaps in available data and ways to fill thegaps– Take advantage of opportunities to “build in” to new datasystems• Stay ahead of “story”8Why can’t I focus onimplementation now and dealwith evaluation later?
Defining Scope10• Set focus– Medicaid only, all health reform activities (state and federal?)• Need to keep the number of topic areas manageable– Access, cost, public health, impact on providers• What are you trying to achieve?– High Medicaid participation rates; good enrollee experience,reduced uninsurance; low rate of coverage gaps• What issues are policymakers most concerned about?– Churn, continuity of coverage, provider capacity to care for newlyuninsured;• Who is the audience?
Choosing Measures• Keep the number of measuresmanageable - prioritize• Choose measures that are directlyrelated to policy goals and levers• Think about near-/medium-/long-termimpacts and include some measures foreach• Include some measures that might be“early success signs” or “early warningsigns”• Consider feasibility - existing data vs.possibility of collecting new data11
Operationalize the Measure• Create a working definition or preferred method forcalculating the measure– e.g., how do you calculate churn?• Defining the “universe”– e.g., population-wide? exchangevs. total market?• Specify the level of detail you want to capture– e.g., disenrollment or disenrollment by reason12
Select Appropriate Data1. Conduct a data scan2. Assess data againsta defined set ofcriteria3. Identify gaps4. Prioritize ways offilling gaps13
Setting Benchmarks and Goals (or not)• Possible benchmarks– Change over time– Defined ideal– Other states– National average• The most useful goals are– Realistic– Specific– Connected to specific actions/strategies and policy priorities• Decisions will influence choices about data sources• Consensus around goals and benchmarks can bechallenging14
Stakeholder Engagement• “Stakeholder” can be defined narrowlyor broadly• Stakeholders can be engaged at anypoint in the process• Best to present stakeholders withsomething to react to• Need clear boundaries on scope andpurpose15
California - Approach• Led by the California HealthCare Foundation (work done by SHADAC)• Development of a set of measures to monitor over time• Geared toward public• Focused on the ACA but limited to 3 topic areas:1. Health insurance coverage (section on public coverage)2. Affordability and comprehensive of coverage3. Access to care• Considerations for measures selection– Measures that reflect major goals and provisions of the ACA– Outcomes rather than implementation process– Relevant/meaningful to policymakers– Interest in measures available at a sub-state level– Data availability• Stakeholders engaged after draft list of measures was developedhttp://www.shadac.org/publications/framework-tracking-impacts-affordable-care-act-in-california16
California - Coverage Measures17Uninsured Public Coverage Employer CoverageDistribution of Insurance CoverageHealth InsuranceExchangePoint in timeEnrollment as Share ofNongroup MarketEmployer participationEmployees in firmsthat offer% EligibleEnrollment trendEmployers payingpenaltyParticipation rateChurningUninsured for a yearor longerUninsured at somepoint in past yearReasons foruninsuranceExempt from mandatePaying penaltyEmployers offeringFamilies with ESIoffer% EnrolledAll familymembers enrolled
California - Affordability &Comprehensiveness of CoverageMeasures18Insurance PremiumsSubsidiesComprehensiveness Financial Burden% of families with highcost burden“Affordable” premiumas % of incomeEmployer coverageTotal premiumEmployee shareSingleFamilyNongroup coveragePer enrolleeEnrollment by benefitlevelESINongroupDeductiblesESI: single, familyNongroup: single,familySingleFamily# receiving premiumand cost sharingsubsidies in exchangeAverage value ofsubsidies
California - Access to Care Measure19Individuals SystemUse of services Barriers to careHas usualsource of careDid not getnecessary care(& reasons)Preventable/avoidable ER visitsSafety netVolume and type ofservices providedby safety net clinicsUncompensatedcareType of placefor usual sourceof care% of physiciansparticipating inpublic programsDifficulty findingprovider thatacceptsinsurance typeDifficulty findingprovider to takenew patientsNot able to gettimelyappointmentAny doctor visitin past yearPreventive carevisit in past yearCounty indigentcare volume andcostAmbulatory caresensitive hospitaladmissionsEmergency roomvisit rate% of physiciansaccepting newpatients, by payer
Maryland - Approach• Led by the Maryland Health Connection (work done bySHADAC)• Development of a set of measures to monitor over time• Geared toward policy makers and the public• Focused on the exchange and limited to 5 coremeasurement categories:– Affordability– Access (includes seamless and non-seamless coveragetransitions)– Consumer Satisfaction– Stability– Health Equity20
Maryland Approach - Continued• Considerations for measures selection– Drawn from data currently produced by other state agencies, datacurrently collected or analyzed by other state agencies orgenerated through exchange– Highly prioritized, no more than 10 measures in each category• Exchange board developed measurement categories andgave feedback throughout the selection of measures• Public comment period after draft list of measures wasdevelopedhttp://marylandhbe.com/wp-content/uploads/2012/12/Performance-Management.pdf21
Too Daunting? Leverage AvailableResources!• Leverage federal funding• Let another agency or division take the lead– Just make sure to stay engaged• Consider outside partners to consult on or lead theseefforts– State universities– Evaluation consultants– Local foundations• No need to remake the whee1– Look at monitoring/evaluation schemes developed by other states(ask your NAMD collogues!)– Utilize data you current collect and use for other purposes (e.g.,operations, reporting)23
Sign up to receive our newsletter and updates atwww.shadac.org@shadacContact InformationElizabeth LukanenSenior Research Fellowelukanen@umn.edu612.626.1537