Building Skills to Monitor & Evaluate Performance & Outcomes

2,872 views

Published on

Presented at the Common Good VT Vermont Nonprofit Conference 2011 by Joy Livingston & Donna Reback, Flint Springs Associates - www.flintspringsassociates.com/

Published in: Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,872
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
67
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Building Skills to Monitor & Evaluate Performance & Outcomes

  1. 1. Building Skills to Monitor and Evaluate Performance and Outcomes Joy A. Livingston, PhD Donna Reback, MSW, LICSW Flint Springs Associates www.flintspringsassociates.com April 14, 2011
  2. 2. Workshop Goals <ul><li>Understand steps needed to monitor & evaluate performance & outcomes in order to market accomplishments and gain funding . </li></ul><ul><li>Learn to develop and use a logic model </li></ul><ul><li>Recognize the investment of time and resources required to monitor and evaluate performance and outcomes </li></ul>
  3. 3. 4 Steps to Monitor and Evaluate <ul><li>Develop a Logic Model </li></ul><ul><li>Gather Information </li></ul><ul><li>Analyze Information </li></ul><ul><li>Use Information </li></ul>
  4. 4. Developing a Logic Model <ul><li>What is a logic model? </li></ul><ul><li>Why develop a logic model </li></ul><ul><ul><li>Provides structure for conversations and decisions about outcomes and performance measures </li></ul></ul><ul><ul><li>Guides development and implementation of monitoring and evaluation activities </li></ul></ul><ul><ul><li>Examines connection between what we do and results </li></ul></ul>
  5. 5. Logic Model Template Inputs/ Resources Activities Performance Measures Outcome Indicators Outcomes What (resources, supports, commitments) do we need to carry out the initiative and achieve desired outcomes? What do we need to do to achieve the desired outcomes? How will we know we did what was needed? How will we know we achieved the expected result? What is the expected result of what we did? Planned work Intended Results
  6. 6. Logic Model- Step 1: Identify and Define Outcomes <ul><li>Agree on outcomes </li></ul><ul><ul><li>Interview individual staff and board members </li></ul></ul><ul><ul><li>Identify areas of consensus and difference </li></ul></ul><ul><ul><li>Hold meeting/retreat to build uniform agreement on outcomes organization is willing to be held accountable for achieving </li></ul></ul>
  7. 7. Why Track Outcomes? <ul><li>Make the case that program is effective </li></ul><ul><li>Focus on activities toward organization’s goals (don’t waste resources on activities not linked to outcomes) </li></ul><ul><li>Help set realistic and realizable goals for organization to be held accountable </li></ul><ul><li>Allow everyone in organization to be part of the conversation about outcomes </li></ul>
  8. 8. Inputs Resources Activities Performance Measures Outcome Indicators Outcomes Clients successfully plan, start and/or grow a business Clients positively contribute to economy Clients achieve financial stability and economic self-sufficiency
  9. 9. Logic Model – Step 2: Identify Outcome Indicators <ul><li>Identify concrete, observable measures </li></ul><ul><ul><ul><li>Community-wide impacts </li></ul></ul></ul><ul><ul><ul><li>Individual client outcomes </li></ul></ul></ul><ul><li>Define measurable benchmarks </li></ul><ul><ul><ul><li>Benchmarks are numerical points which define success within a specified timeframe </li></ul></ul></ul>
  10. 10. Inputs Resources Activities Performance Measures Outcome Indicators Outcomes <ul><li>Clients complete written business plan </li></ul><ul><li>Clients starts new business or expand existing business – business continues </li></ul><ul><li>Client businesses are financially viable (profitable) </li></ul>Clients successfully plan, start and/or grow a business <ul><li>Client businesses contribute jobs </li></ul><ul><li>Client businesses generate tax revenues </li></ul>Clients positively contribute to economy <ul><li>Clients use budget </li></ul><ul><li>Clients improve credit score </li></ul><ul><li>Clients have stable employment </li></ul><ul><li>Clients reduce use of public assistance </li></ul>Clients achieve financial stability and economic self-sufficiency
  11. 11. Outcome Indicators Benchmarks Clients complete written business plan 80% of clients complete business plan within 1 year of intake Clients businesses are financially viable 50% increase in client businesses that are profitable (i.e., revenue less cost > 0) by 2012
  12. 13. Logic Model – Step 3: Determine Activities Leading to Outcomes <ul><li>Examine current activities and services </li></ul><ul><li>Consider if and how each would lead to desired outcomes </li></ul><ul><ul><li>Be willing to eliminate those activities that aren’t linked to achieving outcomes </li></ul></ul><ul><ul><li>Consider incorporating new activities </li></ul></ul>
  13. 14. Inputs Resources Activities Performance Measures Outcome Indicators Outcomes <ul><li>Recruit participants </li></ul><ul><li>Provide 1-to-1 TA & counseling </li></ul><ul><li>Provide training courses </li></ul><ul><li>Client completes written business plan </li></ul><ul><li>Client starts new business or expands existing business – business continues </li></ul><ul><li>Client business is financially viable (profitable) </li></ul>Clients successfully plan, start and/or grow a business <ul><li>Client businesses contribute jobs </li></ul><ul><li>Client businesses generate tax revenues </li></ul>Clients positively contribute to economy <ul><li>Provide 1-to-1 financial and social service counseling </li></ul><ul><li>Referrals within & outside agency </li></ul><ul><li>Clients use budget </li></ul><ul><li>Clients improve credit score </li></ul><ul><li>Clients have stable employment </li></ul><ul><li>Clients reduce use of public assistance </li></ul>Clients achieve financial stability and economic self-sufficiency
  14. 15. Logic Model – Step 4: Identify Performance Measures <ul><li>Consider how you will measure the implementation of agreed upon activities and services </li></ul><ul><ul><li>Determine dosage (measures of amount, extent of services and/or activities performed) </li></ul></ul><ul><ul><li>Determine measures of quality (client satisfaction, fidelity to evidence-based practice) </li></ul></ul><ul><li>Identify benchmarks – numerical measures of successful implementation </li></ul>
  15. 16. Inputs Resources Activities Performance Measures Outcome Indicators Outcomes <ul><li>Recruit participants </li></ul><ul><li>Provide 1-to-1 TA & counseling </li></ul><ul><li>Provide training courses </li></ul><ul><li>Number of clients recruited/retained </li></ul><ul><li>Number of clients receiving TA & types of TA provided </li></ul><ul><li>Number & types of courses offered; number participants; number meeting training objectives </li></ul><ul><li>Participant satisfaction with services </li></ul><ul><li>Client completes written business plan </li></ul><ul><li>Client starts new business or expands existing business – business continues </li></ul><ul><li>Client business is financially viable (profitable) </li></ul>Clients successfully plan, start and/or grow a business <ul><li>Client businesses contribute jobs </li></ul><ul><li>Client businesses generate tax revenues </li></ul>Clients positively contribute to economy <ul><li>Provide 1-to-1 financial and social service counseling </li></ul><ul><li>Referrals within & outside agency </li></ul><ul><li>Number of clients receiving financial counseling </li></ul><ul><li>Number and type of referrals to other social service s </li></ul><ul><li>Clients use budget </li></ul><ul><li>Clients improve credit score </li></ul><ul><li>Clients have stable employment </li></ul><ul><li>Clients reduce use of public assistance </li></ul>Clients achieve financial stability and economic self-sufficiency
  16. 17. Performance Measures Benchmarks Number of clients recruited/retained During 2011, 100 new clients will be recruited; 75% will remain in program Participants satisfied with services 85% of clients report satisfaction with program at 24 month follow-up
  17. 18. Logic Model – Step 5: Identify Inputs and Resources Needed <ul><li>What needs to be in place to support implementation of activities </li></ul><ul><ul><li>Funding </li></ul></ul><ul><ul><li>Policies </li></ul></ul><ul><ul><li>Support of stakeholders </li></ul></ul><ul><ul><li>Skilled, experienced staff </li></ul></ul>
  18. 19. Inputs Resources Activities Performance Measures Outcome Indicators Outcomes Established MBDP and VWBC programs Trained and experienced staff Economic conditions increasing interest in small business <ul><li>Recruit participants </li></ul><ul><li>Provide 1-to-1 TA & counseling </li></ul><ul><li>Provide training courses </li></ul><ul><li>Number of clients recruited/retained </li></ul><ul><li>Number of clients receiving TA & types of TA provided </li></ul><ul><li>Number & types of courses offered; number participants; number meeting training objectives </li></ul><ul><li>Participant satisfaction with services </li></ul><ul><li>Client completes written business plan </li></ul><ul><li>Client starts new business or expands existing business – business continues </li></ul><ul><li>Client business is financially viable (profitable) </li></ul>Clients successfully plan, start and/or grow a business <ul><li>Client businesses contribute jobs </li></ul><ul><li>Client businesses generate tax revenues </li></ul>Clients positively contribute to economy Established MBDP, VWBC and IDA programs Trained and experienced staff Federal and state funds to support programs <ul><li>Provide 1-to-1 financial and social service counseling </li></ul><ul><li>Referrals within & outside agency </li></ul><ul><li>Number of clients receiving financial counseling </li></ul><ul><li>Number and type of referrals to other social services </li></ul><ul><li>Clients use budget </li></ul><ul><li>Clients improve credit score </li></ul><ul><li>Clients have stable employment </li></ul><ul><li>Clients reduce use of public assistance </li></ul>Clients achieve financial stability and economic self-sufficiency
  19. 20. Exercise <ul><ul><li>Individual organization work </li></ul></ul><ul><ul><li>Small group sharing </li></ul></ul><ul><ul><li>Report out and Q&A </li></ul></ul>
  20. 21. After Developing the Logic Model
  21. 22. Gather Information <ul><li>Identify what information needed to measure performance and outcomes </li></ul><ul><li>Does the information already exist? </li></ul><ul><ul><li>If yes, where is it and what form is it in? </li></ul></ul><ul><ul><li>If no, how will it be gathered? </li></ul></ul><ul><li>Who will collect? </li></ul><ul><li>How will it be collected? </li></ul><ul><li>When will it be collected? </li></ul>
  22. 23. Analyze Information <ul><li>Refer to logic model to structure questions for analysis </li></ul><ul><li>Conduct systematic and objective analysis of quantitative and qualitative data </li></ul><ul><li>Review findings with relevant staff/board/stakeholder to determine: </li></ul><ul><ul><li>Did we do what we said we were going to do (performance)? </li></ul></ul><ul><ul><li>How well did we perform? </li></ul></ul><ul><ul><li>Did we achieve desired outcomes? </li></ul></ul>
  23. 24. Use Information <ul><li>Internal - Continuous Quality Improvement </li></ul><ul><ul><li>Improve, change activities and services </li></ul></ul><ul><ul><li>Inform strategic planning </li></ul></ul><ul><li>External – Community Support and Funding </li></ul><ul><ul><li>Market, tell our story </li></ul></ul><ul><ul><li>Build support for our program </li></ul></ul><ul><ul><li>Gain funding </li></ul></ul>

×