2. Colorado Department of Public Health and Environment Strategic Map: 2011-2015 Draft 08/30/11 Align Priorities and Resources to Improve and Sustain Public Health and Environmental Quality A D B E C Create a More Efficient, Effective, Customer-Oriented Department Strengthen the Integrated Public and Environmental Health System Foster a Competent, Empowered Workforce Promote Programmatic Excellence Achieve Targeted Improvements in Colorado’s Winnable Battles Implement the Public Health Improvement Plan Promote Public Health and Environmental Careers Establish Evidence-Based, Data-Driven Strategies Deliver Excellent Customer Service Develop and Implement Strategies to Achieve Winnable Battles 1 Achieve Clear, Specific, Measurable Goals Attract and Retain Talented and Diverse Employees Engage Partners in Programmatic and Policy Decision-Making Employ Lean Methodologies to Improve Business Processes Promote Policies That Support Winnable Battles 2 Value and Share Innovation Align Employee Goals and Roles with Department Priorities Champion Collaboration Among Programs, Departments and Partners Secure Sustainable, Diversified Funding Actively Participate as a Partner in the Community 3 Provide Professional Development and Advancement Opportunities Communicate Winnable Battles to Employees and Identify Linkages to their Work Strengthen Cross-Departmental Collaboration/ Integration Use New and Alternate Technologies to Optimize Employee Performance Create and Support Sustainable, Healthy Communities 4 Meet Regulatory and Statutory Requirements Educate External Partners on Winnable Battles Strengthen Internal and External Communication Develop a Leadership Succession Plan Promote the Value of Public Health and Environmental Protection 5 Consistently Apply Department Policies and Procedures Recognize and Reward Employee Contributions Keep Up with the Speed of Business 6 Influence State HR and Procurement Policies 7 Promote Health Equity and Environmental Justice F Use Performance-Based Measures and Evaluation to Continuously Improve Effectiveness and Prioritize Resources G
3. Topics Insights A bit about Epi and Eval Insights from Karen T. 3 examples of collaboration with EPE Discussion: What would we like to see in a collaborative business model and working relationship with EPE and programs?
4. Good Timing New CDPHE strategic map New leadership in CDPHE & PSD New Colorado Winnable Battles Opportunity to address questions
5. Insights from EPE Strategic Planning… Data driven, evidence-based public health action, with every program, every time Current funding and operations model of EPE is a barrier Our work is more useful when we’ve used a more collaborative model
6. Steps of Evidence-Based Public Health Brownson, RC; Fielding JE; Maylahn CM. Ann. Rev. Public Health 2009.30:189
7. 2. What are the largest problems in the population? Who are at high risk? What is contributing to health? Epi Questions 1. What are the community needs, assets, and values? 3. What potential strategies are relevant? 5. What effective strategies are priorities? 4. What potential strategies are effective? In Colorado? Brownson, RC; Fielding JE; Maylahn CM. Ann. Rev. Public Health 2009.30:189
8. Epidemiology Unit in EPE Indira Gujral Renee Calanan Declan Calanan DOB: 9/29/11 Sue Ricketts, demographer Stephanie Kuhn Ashley Juhl
9. Epi Unit contributes Basic public health science Critique of existing evidence Did the research really work? Was the research done well enough to act on the findings? Assessment of the relationships among assets, protective factors, behaviors, and health in the population Systems thinking Interpretation of real-world data More details at: www.cste.org/competencies.asp
10. Epi Unit contributes Expertise in large data sources: From Health Statistics Section ~ BC, PRAMS, CHS, BRFSS, DC Other state or national sources: WFRS, MEPS, CHA-HDD, MDS, Census, SAIPE, ACS, BSS, YRBS, CIIS,CRCSN, COVDRS,CFPS, FARS, TAR, CSAP, CBI, CDE, PedNSS, NSCH, NS-CSHCN, NHANES, NHIS, TABS, CHKST, NYTS, NIS, NSFG, NVSS, NSDUH, NSRE, HIV/STD, CHP+, COHS, HEDIS, Medicaid, NDI, SHP, HCUP, ESRD
11. We answer the big questions. What matters? What might we do about what matters?
20. Sexual Behavior Among Colorado Adults A CLOSER LOOK: Long Acting Reversible Contra-ceptive Methods (‘02, ’04, ‘05, ‘06, ‘09) Data Source: Behavioral Risk Factor Surveillance System, Colorado Department of Public Health and Environment
21.
22. EPE provides written context for issues “Unintended pregnancy is defined as one that is either unplanned or mistimed (occurred sooner than desired). Women who are intentional about becoming pregnant are more likely to start prenatal care early, and to adopt healthy behaviors during pregnancy. The national Healthy People 2010 Objective 9‐1 is to increase the proportion of intended pregnancies to 70 percent of all births. “ The highest rates of unintended pregnancies are found in two groups: low-income women and young women. In 2008, only 42 percent of births to women on Medicaid were intended, compared to 72 percent of births to women not receiving Medicaid. 6,viii For younger women (ages 15-19) during the same year, only an estimated 36 percent of births were intended. Similarly, only 41 percent of births to women ages 20-24 years old were intended. Intended pregnancies increased significantly after age 24, where 70 percent of births to women ages 25-34, and 67 percent of births to those ages 35 and older were reportedly intended. 6 Reducing unintended pregnancies among young and low-income women would help Colorado reach the Healthy People 2010 objective for intended pregnancy.” Excerpts from Chapter 2, The Health Status of Colorado’s Maternal and Child Health Population, 2010
23.
24.
25. Evaluation in EPE External Evaluation Oversight Internal Evaluation Julie Graves Carsten Baumann Arthur McFarlane Mario Rivera Kerry Thomson
26. Evaluation, defined “Ultimately, evaluation is concerned with asking questions about issues that arise out of everyday practice. It is a means for gaining a better understanding of what we do and the effects of our actions in the context of society and the work environment.” --Russ-Eft, H. and Preskill, D. (2005)
28. What is an Evaluator thinking?! Why…? Why…? When…? How…? Under what circumstance…? More detail & context please! Scan of an Evaluator’s Brain
29. Evaluation Questions Needs Assessment Where are the gaps in the community? What does the community feel is important? Outcome Evaluation Were my strategies effective in achieving the desired outcomes? Formative Evaluation How can we improve the program? How effective are my collaborative partnerships? Process Evaluation How much was provided and with what consistency? Did I reach the target population? Am I hitting my benchmarks? Are my strategies being carried out as planned? Evaluability Assessment What type of evaluation is my program ready for based on what we know? Brownson, RC; Fielding JE; Maylahn CM. Ann. Rev. Public Health 2009.30:189
30. The Evaluation Unit Contributes: Collaboration on logic models Helping write SMART objectives Identifying process and outcome measures to determine if you are meeting goals and objectives Collaboration to decide on an appropriate evaluation focus and design (Evaluability Assessment) Design of data collection forms/procedures Analysis and interpretation of evaluation data
31. Collaboration is Key Understanding of the Health Problem Content knowledge and knowledge about strategies Information about the community Stakeholder relationships and needs Understanding of the Health Problem Understanding of data and the complex determinants of health Scientific method Objectivity Outcome-orientation SMART Objectives + Programs EPE = planning & executing the right analysis, intervention, & evaluation at the right time!
32. Collaboration in PSD We are one division. We are working in horizontal teams, because successful public health is multi-disciplined. 3-way partnership: Epi, Eval, Programs Let’s together work on a collaborative business model for EPE.
35. Maternal, Infant and Early Childhood Home Visiting Program Five federal applications, including statewide needs assessment, state plan for home visiting and a competitive “expansion” application, were submitted EPE = critical program partner from planning to execution RFI to identify existing home visiting programs Needs assessment to identify at-risk communities Prioritization for investments based on county strengths and needs Funding formula to distribute money to prioritized counties Logic Models and Smart Objectives Data collection plan for 6 Benchmark areas with 36 constructs CQI plan and implementation Working closely with our partners, the EB model reps Highly academic evaluation plan developed for competitive application Will analyze and help interpret data; prepare reports
37. Facilitated Discussion How can we create a better collaboration? What would you like to see in a collaborative business model for EPE?
Editor's Notes
Chris introduces the talk.
Chris: CDPHE strategic map with colored boxes and circles that indicate where EPE contributes to operationalizing it.NOTE: A1, D3, E2 are in dark gold, because they are the blue, immediate strategies on the original CDPHE mapGK added C2, C4, E4, E5
Kristin:Introduce selfApologize that Gabriel couldn’t make it today (national conference focused on state policy)Purpose of the talk - Chris asked us to talk about EPE’s contribution and role in PSDStepping stone for a conversation about how EPE might operates in the Division in the future.Briefly walk through topics
Kristin:Reiterate that this is good timing for a conversation such as this.Ask how many people are new to the Division in the last 1 year?Opportunity to:Answer basic questions about who we are and what we doThink critically, together as one division, about how we can best realign our work with the Division priorities
Kristin:EPE just completed a comprehensive, participatory strategic planning process. Ask how many people know about this approach to planning? Share some insights with you from that process.Vision - we really want to contribute in a more significant manner than we have typically been able to or invited to in the past (there have been some exceptions). Business model as a barrier to being as effective, efficient, and elegant as possibleReactionaryBarrier to effective collaboration to align and address shared prioritiesTo put the work of the Epi Unit and Evaluation Unit into perspective, our budget is less than 2% of the PSD budget, we have 12 FTE, and are working on an average of 50 projects each month. Our division has around 150 staff and 60 programs or initiatives.
Barbara:Evidence-based public health is crucial to us in PSD for at least 3 reasons:There is a renewed emphasis on evidence-based public health in the division AND the Department. How many of you took a course recently in EBPH?I think the Colorado Winnable Battles reflects the department’s need for EBPH. If we are going to make measurable change in them, then we need effective strategies.2. The body of evidence for preventing chronic disease and injuries and for improving maternal and child health is still growing and changing.3. PSD is lean, therefore, we need to use our time efficiently on effective strategies.This slide shows one way to describe the steps of EBPH. You have a handout of it. We’ve added example questions that we can answer in each step.On the other side of the handout is a table with more detail. I created the purpose at the top by combining the central theme of the Department’s Strategic Map to the goal of PSD (which is bolded).This slide also shows the steps that the 2 portfolio teams are doing.Currently, there are 2 teams comprised of EPE staff and program staff are doing all of these steps to address:Pregnancy-related depressionImproving Physical Activity in the population.Together we are piloting how best to operationalize these steps in the real world.Now that we have a CDC public health prevention specialist here for 2 year, we plan to expand the number of portfolio teams.Today Kristin and I will use this framework as a way to describe how EPE complements the role that your staff play in EBPH.Both units in EPE work on Steps 1 and 5. In general, the Epidemiology Unit work covers Steps 2-4 and the Evaluation Unit covers Steps 6 and 7.
If you want answers to these types of questions, then you are an honorary epidemiologist or demographer.
I supervise 4 epidemiologists and one demographer. All of us are passionate about taking data to action.
Barbara:We are trained to – recognize public health problems pertinent to the population and to analyze, summarize, and interpret data in such a way to draw valid conclusions. We can recommend evidence-based interventions in response to the data findings and research.So you can see that our work goes beyond quantitative data.
Barbara:The Epi Unit contributes expertise in using existing large data sources from Health Statistics Section and 40 other state and national sources. We are responsible for knowing which data are best to answer which questions or inform which strategies. And we identify the good data from the bad data.We help Programs in PSD develop questions for CHS and BRFSS. If asked:NSDUH = National Survey of Drug Use and HealthNSRE = National Survey of Recreation and Environment by the U.S. Forest ServiceSAIPE = Small Area Income and Poverty EstimatesHEDIS = Healthcare Effectiveness and Information SetMEPS = Medical Expenditure Panel SurveySHP = CDC School Health ProfilesCOHS = Colorado Household Survey (about insurance – HCPF)
Barbara:We use epidemiologic data to guide public health practice and improve health.In other words, we help answer big questions:What matters to the health of the population?What might we do about what matters?It is easier for me to show you than tell you.
(Barbara will scroll through these next 6 slides in 20 seconds.)When we analyze or Health Statistics Section analyzes their data, we create these detailed statistics.So if you submit a data request to Health Statistics Section, this is what you get.These results – as is – are not useful for making quick decisions or for seeing the key finding.(We in the Epi Unit get so excited about all this data.)
And more data …
And more data …
And more data …
And more data …
And more …
… and this is what we give you.Epitakes the data and creates summary graphs for immediate understanding of the health behavior, in this example.
Here, we added the relevant target that highlights how big is the issue and for whom.
By understanding your programs, we provide population-based data as possible evidence of the success of your program.For example, this slide shows the growth in the use of long-acting reversible contraceptive methods in the Colorado population. You can see that there is a large jump in 2009 after the Family Planning program began its statewide initiative to promote and fund such methods.So the trend over time is in the right direction.
This is an example where we combined data from numerous sources (shown in the far right column). The colors show the interpretation of the Colorado results compared to the Healthy People goal.
Here is an excerpt from The Health Status of Colorado’s Maternal and Child Health Population.Given all the data results in the paragraph, what is the implication for the Family Planning Program. We wrote it in the last sentence:“Reducing unintended pregnancies among young and low-income women would help Colorado reach the Healthy People 2010 objective for intended pregnancy.”
We create comprehensive reports like this, about one a year. This report isThe Weight of the State: 2009 Report on Overweight and Obesity in Colorado.Some results from this report were also used for: press releases, the community assessment and quantifying the issue for the physical activity portfolio team the obesity talking points in Google docsto describe the high risk groups listed in the factsheet on obesity for the Colorado Winnable Battles
Reports like this provide in-depth analysis of numerous issues, with appropriate graphics and interpretation.We had to formulate a data analysis plan based on the literature and what we know about Colorado.We had to take the detailed & complex results and distill them into the most meaningful ones.And then we had to summarize them visually and in writing so that the complex information is most readily accessible ANDThat it tells both an accurate and complete story.
Context:Currently, 6 people focused on evaluation in EPE.There are lots of ways to categorize evaluation projects. For the purpose of managing the evaluation workload in a division of 200 employees, we distinguish two types of evaluation – internal (Kristin) and external (Carsten)Four internal eval staff – work on a variety of projects (simple to complex)
Evaluation is a practice/discipline that can be applied quite broadly in an organization. It’s focused on your specific program and the activities you carry out as part of your effort.
People think about evaluation, they think about it in terms of measuring outcomes – did we achieve our goal or did we not achieve it? Difference between public health surveillance and evaluationSurveillance monitors the broad context of public health (how are we doing on the Winnable Battles)Evaluation is focused on programs and their more immediate results in that broad context.
Evaluators work hard to understand where your program is, where it’s been, and where it is going, so that we can help you ask the right questions at the right time. Because evaluation is only as good as the question that’s asked. In a LEAN environment, we have a huge obligation to make sure that we are evaluating programs and processes in a way that help us make better decisions.
We can apply evaluation to every step in the Evidence-based public health model. The type of evaluation and the question you might changes depending on where you are.
Recap how an evaluator can help with good EBPH.
Epidemiology and evaluation can’t be done in isolation. All of us in PSD bring a valuable skill setof skills and expertise to the table.EPE cannot do it alone, if PSD wants to plan & execute the right analysis, strategy, and evaluation at the right time!
Collaboration is important in PSD because:We are one division & becauseSuccessful public health is multi-disciplinedIt is the best use of everyone’s time and talent in a lean division where less than 10% of the division funding stays in the division.Perfect recipe for improved collaboration.
Karen T. speak about her insights from her work with EPE
Placeholder for Celeste’s Slides
Placeholder for Mary’s Slides
Placeholder for Gina’s slides
Karen facilitates, Barbara and Kristin record on flipcharts (?)