Building Skills to Monitor & Evaluate Performance & Outcomes

Building Skills to Monitor and Evaluate Performance and Outcomes Joy A. Livingston, PhD Donna Reback, MSW, LICSW Flint Springs Associates www.flintspringsassociates.com April 14, 2011

Workshop Goals
Understand steps needed to monitor & evaluate performance & outcomes in order to market accomplishments and gain funding .
Learn to develop and use a logic model
Recognize the investment of time and resources required to monitor and evaluate performance and outcomes

4 Steps to Monitor and Evaluate
Develop a Logic Model
Gather Information
Analyze Information
Use Information

Developing a Logic Model
What is a logic model?
Why develop a logic model
Provides structure for conversations and decisions about outcomes and performance measures
Guides development and implementation of monitoring and evaluation activities
Examines connection between what we do and results

Logic Model Template Inputs/ Resources Activities Performance Measures Outcome Indicators Outcomes What (resources, supports, commitments) do we need to carry out the initiative and achieve desired outcomes? What do we need to do to achieve the desired outcomes? How will we know we did what was needed? How will we know we achieved the expected result? What is the expected result of what we did? Planned work Intended Results

Logic Model- Step 1: Identify and Define Outcomes
Agree on outcomes
Interview individual staff and board members
Identify areas of consensus and difference
Hold meeting/retreat to build uniform agreement on outcomes organization is willing to be held accountable for achieving

Why Track Outcomes?
Make the case that program is effective
Focus on activities toward organization’s goals (don’t waste resources on activities not linked to outcomes)
Help set realistic and realizable goals for organization to be held accountable
Allow everyone in organization to be part of the conversation about outcomes

Inputs Resources Activities Performance Measures Outcome Indicators Outcomes Clients successfully plan, start and/or grow a business Clients positively contribute to economy Clients achieve financial stability and economic self-sufficiency

Logic Model – Step 2: Identify Outcome Indicators
Identify concrete, observable measures
Community-wide impacts
Individual client outcomes
Define measurable benchmarks
Benchmarks are numerical points which define success within a specified timeframe

Inputs Resources Activities Performance Measures Outcome Indicators Outcomes
Clients complete written business plan
Clients starts new business or expand existing business – business continues
Client businesses are financially viable (profitable)
Clients successfully plan, start and/or grow a business
Client businesses contribute jobs
Client businesses generate tax revenues
Clients positively contribute to economy
Clients use budget
Clients improve credit score
Clients have stable employment
Clients reduce use of public assistance
Clients achieve financial stability and economic self-sufficiency

Outcome Indicators Benchmarks Clients complete written business plan 80% of clients complete business plan within 1 year of intake Clients businesses are financially viable 50% increase in client businesses that are profitable (i.e., revenue less cost > 0) by 2012

Logic Model – Step 3: Determine Activities Leading to Outcomes
Examine current activities and services
Consider if and how each would lead to desired outcomes
Be willing to eliminate those activities that aren’t linked to achieving outcomes
Consider incorporating new activities

Recruit participants
Provide 1-to-1 TA & counseling
Provide training courses
Client completes written business plan
Client starts new business or expands existing business – business continues
Client business is financially viable (profitable)
Provide 1-to-1 financial and social service counseling
Referrals within & outside agency
Clients use budget

Logic Model – Step 4: Identify Performance Measures
Consider how you will measure the implementation of agreed upon activities and services
Determine dosage (measures of amount, extent of services and/or activities performed)
Determine measures of quality (client satisfaction, fidelity to evidence-based practice)
Identify benchmarks – numerical measures of successful implementation

Number of clients recruited/retained
Number of clients receiving TA & types of TA provided
Number & types of courses offered; number participants; number meeting training objectives
Participant satisfaction with services
Number of clients receiving financial counseling
Number and type of referrals to other social service s
Clients use budget

Performance Measures Benchmarks Number of clients recruited/retained During 2011, 100 new clients will be recruited; 75% will remain in program Participants satisfied with services 85% of clients report satisfaction with program at 24 month follow-up

Logic Model – Step 5: Identify Inputs and Resources Needed
What needs to be in place to support implementation of activities
Funding
Policies
Support of stakeholders
Skilled, experienced staff

Inputs Resources Activities Performance Measures Outcome Indicators Outcomes Established MBDP and VWBC programs Trained and experienced staff Economic conditions increasing interest in small business
Number of clients recruited/retained
Number of clients receiving TA & types of TA provided
Number & types of courses offered; number participants; number meeting training objectives
Participant satisfaction with services
Clients positively contribute to economy Established MBDP, VWBC and IDA programs Trained and experienced staff Federal and state funds to support programs
Number of clients receiving financial counseling
Number and type of referrals to other social services
Clients use budget

Exercise
Individual organization work
Small group sharing
Report out and Q&A

After Developing the Logic Model

Gather Information
Identify what information needed to measure performance and outcomes
Does the information already exist?
If yes, where is it and what form is it in?
If no, how will it be gathered?
Who will collect?
How will it be collected?
When will it be collected?

Analyze Information
Refer to logic model to structure questions for analysis
Conduct systematic and objective analysis of quantitative and qualitative data
Review findings with relevant staff/board/stakeholder to determine:
Did we do what we said we were going to do (performance)?
How well did we perform?
Did we achieve desired outcomes?

Use Information
Internal - Continuous Quality Improvement
Improve, change activities and services
Inform strategic planning
External – Community Support and Funding
Market, tell our story
Build support for our program
Gain funding

Building Skills to Monitor & Evaluate Performance & Outcomes

by Lauren-Glenn Davitian on Apr 14, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Building Skills to Monitor & Evaluate Performance & Outcomes — Presentation Transcript