Health Start Interconception Care Learning Collaborative_Johannie Escarne_4.2...
Stewart_CapstonePresentation_2015 Dec
1. Resource &Technical Assistance
Recommendations:
2015 Formative Evaluation of the
Colorado Health Assessment and Planning System
(CHAPS)
December 11, 2015
Haley J. Stewart, MPH candidate
Community & Behavioral Health
Practicum and Capstone Preceptors: Cambria Brown, MPH, Alison Long, MPH
Site: Office of Planning, Partnerships & Improvement, Colorado Department of Public
Health and Environment
Advisor: Holly Wolf, PhD, MSPH
2. Acknowledgements
• Local public health agencies and their staff
• Preceptors Alison Long, MPH & Cambria Brown, MPH
• Colorado of Department of Public Health and Environment, Office of
Planning, Partnerships & Improvement staff
• CDPHE evaluators Abby Laib, MS and Leah Brooke, PhD and staff
Heather Baumgartner, MSS and Laurie Schneider, MPH
• Holly Wolf, PhD, MSPH
• my family
ThankYou!
3. Colorado HealthAssessment and Planning System
CHAPS
Colorado
2008 Public Health Act
SB08-194
8 phase assessment
and planning process
Resources
and
technical assistance
• Public health
improvement plan
(PHIP)
• Community health
assessment (CHA)
• Plan development on
a 5-year cycle
I: Plan the process
II: Engage stakeholders
III: Community health
assessment
IV: Capacity assessment
V: Prioritize health issues
VI: Develop public health improvement plan
VIII: Inform the state plan
VII: Implement, monitor, promote the plan
Resources and technical
assistance examples:
• Action Plan template
• Evaluation Plan template
• Defining the scope of the
evaluation
• Interpreting data
4. Purpose
• Assess the efficacy of CHAPS resources and technical assistance
provided by the OPPI to generate improvement recommendations for
the next assessment and planning cycle in start in 2016.
Utilization-Focused Evaluation (UFE)
• To ensure usability of results, the intended primary users were
involved throughout the process.
Research question
What are the strengths, weaknesses, and gaps in
the tools, resources, and OPPI technical
assistance?
Formative Evaluation
6. Primary Findings
• Technical assistance can also
be a place to offer feedback
• Need for TA contacts to be
more accessible
• Different methodologies to
choose priority issues
• CHAPS (former) website was
not intuitive
“I think it would have been a richer
process for us and the [OPPI] if we
were more closely associated
with them and talking with them...” pilot
agency
Weaknesses in technical assistance
• CHAPS phases are “digestible”
• Resource materials are
comprehensive
• Templates are “scalable”
“I just really appreciate the ability to
make a one-size-fits-all [process] -
from an El Paso size health
department down to a Hinsdale...it is
so scalable regardless of the
size of the health department.”
small, rural agency
Strengths in resources
7. Current assessment and
planning challenges
• Monitoring and
evaluation of the public
health improvement plan
“We haven’t really put an evaluation
plan together. I’m just using my progress
update on the goals and strategies document
to address whether or not we have met
certain goals and objectives.” small, rural agency
Next cycle needs
• Data Collection, analysis, and
interpretation
• Alignment
• Evidence-based resources
and strategies
• Project management
“I think that we could absolutely
benefit from more training in data
collection. I think that is
something…[that] can take us three hours
whereas for someone else, may just take a
few minutes.” public health nurse
Primary Findings
8. Impact of the CHAPS process on capacity
• Flexible process
• Community-based
• Organized
• Validating
“ I think that [CHAPS] is helping get everybody
on the same page to … go through a
process with community stakeholders
to determine evidence-based strategies
that they can work together on…build the
capacity of lots of public health agencies to do
this…more systematically…and
consistently.” small, rural agency
Primary Findings
9. “…the Public Health Act of
2008 moved public health
in Colorado from a
patchwork quilt to a
blanket…this public health
planning process moved us
another step closer to that
blanket…to actually doing what
the community needed.”
Significance
for the Colorado public health system
11. Best Practice Resources
o Public health improvement plan evaluation and monitoring support,
training, and resources
Alignment & Partnerships
o Non-profit hospitals community health needs assessment (CHNA)
partnerships
o Public Health Accreditation Board
Technical Assistance
o Categorized technical assistance from OPPI
Integration & Inclusion
o Environmental Health in the CHAPS process
o Tribal health resources and support
Recommendations
12. References
1. Colorado Senate Bill 194, Concerning public health, and making an appropriation in connection therewith (2008). Retrieved from
https://www.colorado.gov/pacific/sites/default/files/OPP_Senate-Bill-08-194.pdf
2.Colorado Department of Public Health and Environment. (2015) Colorado Health and Assessment Planning System: Phase VII. Retrieved
from https://www.colorado.gov/pacific/cdphe-lpha/phase-vii-implement-promote-and-monitor-plan
3. U.S. Census. (2010). Rural, urban by county 2000-2010. Retrieved from http://dola.colorado.gov/demog-cms/content/census-data
(12/5/15)
4. U.S. Census. (2010). American FactFinder. Profile of General Demographic Characteristics: 2000 Census 2000 Summary File 1. Retrieved
from http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=bkmk
5. U.S. Census. (2012). 2010 Census Brief. The American Indian and Alaska Native Population: 2010. Retrieved from
http://www.census.gov/prod/cen2010/briefs/c2010br-10.pdf
6. Colorado Commission of Indian Affairs. (2015). Initiatives. Health and Wellness: Colorado Commission of Indian Affairs Health and
Wellness Roundtable August 19-20, 2013 Ignacio, CO. Retrieved from https://sites.google.com/a/state.co.us/colorado-commission-of-
indian-affairs/committees/health-and-wellness
Figures and images:
Slide 5- Map, Colorado Department of Public Health and Environment (CDPHE), Communication and Graphics department
Slide 9 – Quilt image, retrieved from http://pileofabric.com/blogs/modern-quilting/15173281-giveaway-friday-concerto-pattern-modern-
patchwork-magazine
Slide 10 – Map, CDPHE CHAPS website, Colorado Winnable Battles: Local Strategy Maps, retrieved from
http://www.chd.dphe.state.co.us/chaps/Documents/WBMaps/Obesity6.swf
Slide 11- Photo image, retrieved from https://c2.staticflickr.com/6/5056/5543983845_530e652221.jpg
Good afternoon.
I helped lead the 2015 Formative Evaluation of the Colorado Health Assessment and Planning System, or CHAPS during my public health practice practicum and capstone with the Colorado Department of Public Health and Environment’s Office of Planning, Partnerships and Improvement, or OPPI.
My capstone and this presentation represents the qualitative component.
Before I begin, I would like to thank the local public health agencies and their staff members for contributing their insights, along with my preceptor, Cambria Brown and the OPPI staff, my advisor Holly Wolf and importantly, my family.
Bill: https://www.colorado.gov/pacific/sites/default/files/OPP_Senate-Bill-08-194.pdf
In order to share my evaluation recommendations, I’d first like to explain what CHAPS is and why this was an opportune time to evaluate.
CHAPS is based on requirements from the 2008 Public Health Act. The Act mandates each local public health agency to develop a public health improvement plan based on a community health assessment every 5 years. These requirements reflect nationally-established best practices to ensure quality delivery of public health services1.
Subsequently, CHAPS was created by state and local stakeholders to provide an 8-phase public health improvement process by which to achieve the mandated requirements.
As liaisons between local and state agencies, OPPI is responsible for maintaining and delivering CHAPS. This specifically entails resources and technical assistance, which are included within the steps of each phase and are the heart of this evaluation2.
The next assessment and planning cycle begins later in 2016, and all 55 agencies, representing 64 counties, are expected to participate, making this an opportune time to evaluate and update the original set of resources and technical assistance to keep CHAPS viable, relevant and sustainable for its stakeholders.
To that end, the overall purpose of the formative evaluation was to assess the efficacy of resources and technical assistance to generate improvement recommendations for the next cycle.
Importantly, to ensure usability of the results, I took a utilization-focused approach, which means that the intended users of the recommendations – mainly the OPPI staff – were involved throughout the process.
This better prepared them to use the results to the primary research question: what are the strengths, weaknesses and gaps in the tools, resources and technical assistance?
Image: CDPHE
CHAPS is a shared process, so naturally the evaluation was too.
I collected primary data through 15 1-hour key informant phone interviews and used a 27-item semi-structured interview guide.
The interviews were an open invitation to public health planners,
which resulted in a fairly representative sample of rural, frontier, and urban counties, as well as 2 regional partnerships.
Strengths and weakness were the primary findings of this evaluation.
Strengths were primarily found in the CHAPS resources: the phases were digestible and the resources were comprehensive. One interviewee like that the templates were scalable regardless of the size of the health department.
Conversely, weaknesses were more apparent in technical assistance and the communication of the resources. One interviewee explained that she would have liked better feedback during technical assistance. User-end perspectives also captured that agencies were using different methodologies to prioritize health issues, which ultimately weakens their comparison across the state. Another interviewee felt that it would have been a richer process if the agencies and the OPPI were more closely associated.
To provide context, I also inquired about an agency’s current assessment and planning challenges and their staffs’ needs for the next CHAPS cycle.
A current challenge across all 15 interviews was monitoring and evaluation of their public health improvement plans. To which one remarked that “we haven’t really put an evaluation plan together” and was substituting project progress for evaluation.
Secondly, interviewees perceived needs identified areas to consider that could have been easily forgotten, such as data collection. There is significant flux in agency staff members working on assessment and planning, which means their capacity levels are changing and their strengths may not always include fundamental public health skill sets required to do this work.
Lastly, to find to the meaning behind the interviewees’ responses, I ended each interview by asking about the impact of the CHAPS process on their agency’s capacity.
Despite challenges and weaknesses discussed, they expressed the benefit of a flexible process that they could tailor to their agency that validated their work at the local level as well as statewide. On interviewee captured the benefits of involving the community more in systematic, consistent and inclusive process.
This evaluation focused on minute details, however the significance of keeping CHAPS resources and technical assistance viable, relevant and sustainable supports a unifying public health service that impacts all Coloradans in a historically ‘home-rule’ state.
An interviewee offered the metaphor that this work has changed the state’s population health efforts from a patchwork quilt approach to a more uniform and stronger blanket.
http://www.chd.dphe.state.co.us/chaps/Documents/WBMaps/Obesity6.swf
To continue the analogy, the state’s efforts behind addressing priority health issues, such as obesity pictured in this map, is strengthened by a more coordinated effort.
It is the evidence-based approach of CHAPS that empowers local public health agencies to achieve such convergence without sacrificing unique, county-specific needs and assets.
Based on the strengths, weaknesses, challenges, needs and significance, I propose the following 6 recommendations.
BP
First, best practice resources around monitoring and evaluation need to be provided. These activities are pivotal to the assessment and planning process because this is how an agency can determine if they are having any impact, what is working and what is not.
A
Secondly, let’s look for alignment and partnerships. Non-profit hospitals are currently conducting a similar assessment and planning process on a 3-year cycle.
CHAPS is the way to support hospital and local public health agency partnerships so that together these two sectors can optimize their strengths and synergistically support population health within their jurisdictions.
Also, many local public health agencies are pursuing accreditation, a process that is misconceived to be separate from the CHAPS process.
The two are actually quite aligned, which can be better reflected in the resources and technical assistance.
TA
Next, categorized technical assistance.
Although interviewees supported a flexible process that was not prescriptive, they also want and need technical assistance that is tailored to their needs.
Focus is often put on where most of the population lives in the metro, front range region; however 53% of Colorado counties can be defined as partial to completely rural3. And rural geographies are known to present unique barriers to public health and health care services.
I
Lastly, integration and inclusiveness.
First, take the opportunity to integrate environmental health into the CHAPS process. Clean air and clean water, for example, are two health issue areas that are prioritized at the local level. They are also two of Colorado’s Winnable Battles, and representative of both positive and negative contributors to population health.
Secondly, I advocate for tribal health to become a part of the CHAPS process because of this population’s known health disparities. American Indians account for a small portion of the population4; however it is increasing5. (45% from 2000-2010) and two tribal lands are located within Colorado – the Southern Ute and Ute Mountain Ute.
Inter-governmental integration exists for health and wellness6 and importantly, the Center for American Indian and Alaskan Native Health Center is located directly on this campus, representing additional expertise in supporting the overall health of this portion of Colorado.
Collectively, best practice resources, alignment, partnerships, categorical support, integration and inclusion recommendations can update CHAPS resources and technical assistance so that the process remains viable, relevant and sustainable for its stakeholders.
http://www.ucdenver.edu/academics/colleges/PublicHealth/About/news/CommunityNews/Pages/The-Colorado-Trust-Chair-in-American-Indian-Health.aspx
