Case Study: Plummer Youth Promise, How to Measure Outcomes in Apricot Software

Case Study: Plummer Youth Promise
How to Measure Outcomes in Apricot Software
© 2020 Sidekick Solutions LLC | © 2020 Plummer Youth Promise

On the webinar today…
© 2020 Sidekick Solutions LLC | © 2020 Plummer Youth Promise2
Jeffrey Haguewood
Apricot Software Consultant
Sidekick Solutions
jeff@sidekicksolutionsllc.com
Sarah Morrill
Director of Outcomes and Evaluations
Plummer Youth Promise
smorrill@plummeryouthpromise.org

How to design a program and
outcomes model, and then set up a
data system that enables the model?

Agenda
• Part 1: Designing the program and outcomes model
• Part 2: Assessing gaps in the initial data system’s design
• Part 3: System redesign for integration with the model
4
Today’s presentation will be a mix of strategy and technical.
© 2020 Sidekick Solutions LLC | © 2020 Plummer Youth Promise

Founded in 1854

Key questions
• How well did we do? OUTCOMES
• How satisfied are clients after working with us? SATISFACTION
• How much did we do? OUTPUTS
• What difference do we make? IMPACT
• How true to the model are our practices? FIDELITY
• How do we compare? COMPARISON

Permanency
7 © 2020 Sidekick Solutions LLC | © 2020 Plummer Youth Promise
An enduring family relationship that:
• Is safe and lifelong;
• Offers legal rights and social status of full family membership
• Provides for physical, emotional, social, cognitive, and spiritual well
being; and
• Assures life long connections to birth and extended family, siblings
and other significant adults, family history and traditions, race and
ethnic heritage, culture, religion, and language.
Call to Action: An Integrated Approach to Youth Permanency and Preparation for Adulthood. Casey Family Services in collaboration with
California Permanency for Youth Project, Casey Family Programs, and The Jim Casey Youth Opportunities Initiative

The intervention and outcome model
comes first, preceding the data
system for tracking the model.

How we got started
• We started with our experience, anecdotes and stories
• We knew families were key to permanency
…but we had little data to support our beliefs

Our approach
1. External research
2. Internal discovery
…data system would come later

Data gathering questions
• What problem are we trying to
address?
• What does best practice research tell
us?
• How do we stack up?
• What makes our clients the same or
different?
• What makes our services different?
• What do we believe about our
program participants?
• What does success look like for
program participants?
• Which activities/practices/interventions
drive success?
• How do we know success when we
see it?

What we believe it takes…
• to have permanency through a safe emotionally secure
parenting relationship with a life-long, preferably, legal family;
• to be prepared by having the skills and support to meet
his/her/their physical, emotional, educational, and economic
needs; and
• to be connected to his/her/their community by having a safe
place to live, a sense of belonging and a chance to positively
contribute to the community.

Takeaway: Prioritize research and discovery
• What is known about the problem we are trying to solve?
• External: What does best practice tell us?
• Internal: What do we know about what we already do?
• Create a “body of knowledge” of your practice area

Takeaway: Develop strong support

Takeaway: Merge evidence and performance
Performance-based:
1. to collect data that measures program outcomes and the
indicators that lead to positive outcomes
2. and to demonstrate, over time, continuously improving
trends in performance
Performance-based principles can lead to innovation.

Takeaway: Universal + document heavily
• Universal intake and discharge (with supplements)
• Shared treatment planning process
• Consistent assessment scales (the 4-point “Matrix”)
• All-staff and all-shift progress note template
• Documentation of the model is key 

Data system
Keys:
 Easy to use
 Transition from paper
 Budget friendly

We wanted Apricot to…
• Support compliance requirements
• Collect, manage, and report on outcomes data
• Reinforce our clinical practice
…but how do we configure it without concrete examples?

Challenges with initial Apricot design
• Difficult to track multiple enrollments per youth
• Overwriting historical status to update current status
• Unable to correlate data without heavy lifts in reporting
• Forms were getting larger, staff complaints getting louder

Apricot database “spec”

Why map Apricot’s data model?
• What reports are possible or not possible?
• How are those reports configured?
• How efficient it is to build a report?
• Can we report on outcomes?

Initial observations
• Unable to scale system with new programs long term
• No program enrollment for “continuity of service”
• No workflow systems to support clinical practice
• Data needed export for reporting, highly inefficient

Existing vs. Vision ERD

The decision of whether or not to
proceed with redesign of our Apricot
from 1.0 to 2.0 was a significant
decision for us.

Our decision was between two options
1. Stay with current system
2. Redesign for a new system 
o Budget constraints
o Timeline considerations
o Fears + goals

Priorities for redesign (the best path)
• Staggered implementation approach (easier to digest)
• No data migration, start fresh (lower one-time cost)
• Build from conclusions of spec project (reduce fears and
address goals)

Takeaway: Define what’s what
• Define the current state or status quo
• Define a vision of the future opportunity
• Compare the two to develop the roadmap
If the assessment validates your challenges with
opportunities for resolution, then the conclusions of the
assessment are relevant in next steps (i.e. an action plan).

Takeaway: Run a value equation
• Time
• Cost
• Accuracy (decision making)
• Energy (stress and focus)
Learn more at https://www.sidekicksolutionsllc.com/apricot/evaluating-for-purchase#the-business-case-
for-new-social-work-case-management-software

Takeaway: Data migration isn’t only option
Redesign 12-months
(end of split reporting)
24-months
(complete data set)
• Archive in place, view-only for reference
• Transition on fiscal or calendar year break
• Data always there to migrate if needed in the future

The redesign of Apricot must enable
our practice and provide for a
complete data set to measure
progress and outcomes.

Feature: Universal system design
• Referral tracking with source and eligibility reporting
• Face sheet with standardized demographics and profile
• Consolidated intake and discharge (enrollment) 
• Single Tx Plan, progress note template, and assessment

Feature: Program enrollment
• Multiple, simultaneous, and repeat enrollments 
• Track continuity of service (transitions)
• Global reporting across programs
• Operational consistency
• Scalable architecture as programs change 

Feature: Enrollment supplements
• Wizard linked form (or workflow tool)
• 1-to-1 linking (no multiples in reports)
• Data quality where incomplete

Feature: Workflow design for the model

Feature: Active status dashboard
• Open cases by program and staff assigned
• Progress review/updates to do by staff assigned
• Tx plans to do by staff assigned
• Outcomes assessments to do by staff assigned

Feature: Indicators at interval
• Behaviors
• Services
• Milestones
• Domain area ratings (“Matrix”)
• Goal status

31
2

Aggregate reporting and interval correlation

Feature: Youth and program-specific data
• Team form (relationships)
• Global, correlate with all programs
• Track entities as T2 form

Feature: Youth Summary Report (YSR)

Feature: Leadership team reports
• Active youth as of today
• Discharged youth with date filter
• Referrals (incoming) with date filter
• Youth that participated in range

Feature: Printable templates

Feature: Case reviews and accreditation

Our Apricot by the numbers…
• 49 active users
• 6 programs with shared design
• 100% staff engagement
• Aligned with model

How we know the culture has changed…

Positives
• Apricot design reinforces intervention and outcomes model
• Fewer reports all filtered by program, more efficient
• More time available on capacity building
• Adding new programs is an overlay on current design
• User-driven data quality reports and quality assurance

Areas of improvement
• Vocabulary and category changes in field sets
• Measured approach when planning for new programs
• Staff onboarding and ongoing training
• Updating data policies and procedures for accreditation

Next steps
• Supervisory tools to further support clinical practice
• Build out of more report tools for analysis
• Refining case study practice for accreditation
• Itemize financial benefit of our intervention
• Grow data set for evaluation studies and peer collaboration

What we learned in this process
Apricot is most valuable when specifically using it as a tool to measure your vision.
Step 1: Articulate who you are
Step 2: Understand what you do
Step 3: Check in with stakeholders both internal and external
Step 4: Write down what you hope to report out at the end of the quarter/year/decade
Step 5: Articulate program activities that implement the mission and reach the vision
Step 6: Document behavior changes indicating success
Step 7: Check in with staff and map strategic priorities
Step 8: Document required reporting for compliance, funding, or any others
Step 9: Write logic models by program
Step 10: Design a data model, reporting platform, and client case management system reflective of Steps 1-9

We invested more in our data,
analytics, and people systems than we
originally planned, but have found it
necessary to progress in our work.

We’re continuously learning
• How well did we do? OUTCOMES
• How satisfied are clients after working with us? SATISFACTION
• How much did we do? OUTPUTS
• What difference do we make? IMPACT
• How true to the model are our practices? FIDELITY
• How do we compare? COMPARISON

Thank you for joining us today!
Recording and slides will be sent to you via email.

Open door to connect…
Jeffrey Haguewood
Apricot Software Consultant
Sidekick Solutions
jeff@sidekicksolutionsllc.com
Sarah Morrill
Director of Outcomes and Evaluations
Plummer Youth Promise
smorrill@plummeryouthpromise.org

Questions?

Case Study: Plummer Youth Promise, How to Measure Outcomes in Apricot Software

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Case Study: Plummer Youth Promise, How to Measure Outcomes in Apricot Software

Similar to Case Study: Plummer Youth Promise, How to Measure Outcomes in Apricot Software (20)

More from Jeffrey Haguewood

More from Jeffrey Haguewood (8)

Recently uploaded

Recently uploaded (20)

Case Study: Plummer Youth Promise, How to Measure Outcomes in Apricot Software