United Way or Erie County - Programs, Program Monitoring and Evaluation, and Logic Models
•Download as PPTX, PDF•
0 likes•1,094 views
Caroline Taggart, Senior Evaluator, was invited by the United Way of Buffalo & Erie County to present at the organization’s Board Leadership Training program. Caroline’s presentation covered the importance and general tenets of Program Monitoring and Evaluation, with an emphasis on questions non-profit organization’s Board Members can ask to encourage their organization’s engagement in these activities to ensure quality program delivery and maximum impact.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to United Way or Erie County - Programs, Program Monitoring and Evaluation, and Logic Models
Similar to United Way or Erie County - Programs, Program Monitoring and Evaluation, and Logic Models (20)
Recently uploaded
Recently uploaded (20)
United Way or Erie County - Programs, Program Monitoring and Evaluation, and Logic Models
- 1. Program and Program Outcomes Program Monitoring and Evaluation
- 2. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 3. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 4. What does this organization do that you respect?
- 5. Prove that you are making a difference!
- 12. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 13. Program A planned, coordinated group of activities and/or services, often for a specific purpose or to achieve a goal.
- 14. The systematic and continual documentation of key aspects of a program in order to assess whether the program is being implemented as intended. Program Monitoring Quality Improvement Process Monitoring Fidelity of Implementation Performance Management Process Evaluation
- 15. The systematic method for collecting, analyzing, and using information to answer questions about a program’s effectiveness. Program Evaluation Program Outcomes Impact Evaluation Outcome Evaluation
- 16. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 17. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 18. What are our goals?
- 19. Strategic Plan
- 21. My Goal: Healthy Heart
- 22. What are the needs? How do we identify them?
- 23. Needs Assessment What do our participants need to reach the goal? What are the barriers to reaching the goal?
- 25. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 26. What do we do? What results do we expect?
- 27. Theory of Change/ Logic Model
- 28. IF THEN
- 30. Based on Needs and Goals
- 31. Inputs Activities Outputs Outcomes Impact
- 32. Inputs Activities Outputs Outcomes Impact What do we need to implement the program? Funding Space Curricula Training Participants Staff
- 33. Inputs Activities Outputs Outcomes Impact What activities need to be offered to help us achieve our intended results? Tutoring Curricula Find Homes Distribute Food Provide Counseling Education
- 34. Inputs Activities Outputs Outcomes Impact How will we know that our activities happened? # of Hours # of Participants # Distributed # Housed # of Counseling # Completed # Contacted
- 35. Inputs Activities Outputs Outcomes Impact What results do we expect? Change in Knowledge Change in Skills Change in Attitude Change in Behaviors
- 36. Inputs Activities Outputs Outcomes Impact What do we ultimately want to change? Conditions in Life, such as: Graduation College Entry Stable Housing Earnings Obesity
- 37. Inputs Activities Outputs Outcomes Impact LOGIC THENIF
- 38. Inputs Activities Outputs Outcomes Impact Eat less fat Vigorous Physical activity Lose Weight Lower LDL Healthy Heart # meals low sat. fat # mins vig. activity /wk Healthy Recipes Sneakers Gym Member ship
- 39. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 40. How do we know if we’re on track?
- 41. How do we know if we achieved our results?
- 42. Inputs Activities Outputs Outcomes Impact Program Monitoring
- 43. Inputs Activities Outputs Outcomes Impact Eat less fat Lose Weight Lower LDL Healthy Heart # meals low sat. fat # mins vig. activity /wk Healthy Recipes Sneakers Gym Member ship Vigorous Physical Activity
- 44. Inputs Activities Outputs Outcomes Impact Program Evaluation
- 45. Inputs Activities Outputs Outcomes Impact Eat less fat Vigorous Physical Activity Lose Weight Lower LDL Healthy Heart # meals low sat. fat # mins vig. activity /wk Healthy Recipes Gym Member ship Sneakers
- 46. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 48. What do we measure? What should we measure?
- 49. Inputs Activities Outputs Outcomes Impact
- 50. Complex Simple
- 51. Inputs Activities Outputs Outcomes Impact # of Attendees Hours of Counseling Hours by Type of Counseling
- 54. Inputs Activities Outputs Outcomes Impact # meals low sat. fat
- 56. Inputs Activities Outputs Outcomes Impact Change in Knowledge Increase Positive Behaviors
- 57. Simple: Surveys
- 59. Inputs Activities Outputs Outcomes Impact Lose Weight Lower LDL
- 60. Complex: Blood Test Simple: Weight
- 61. Inputs Activities Outputs Outcomes Impact Increased Graduation Reduced Poverty Less Teen Pregnancy Increased Earnings Reduction in Obesity
- 64. Inputs Activities Outputs Outcomes Impact Healthy Heart
- 66. Inputs Activities Outputs Outcomes Impact Eat less fat Walking Biking Lose Weight Lower LDL Healthy Heart # meals w/out bacon # days walked/ biked # hrs walked/ biked Healthy Recipes Sneakers Bike When should we collect and measure?
- 67. Inputs Activities Outputs Outcomes Impact Eat less fat Vigorous Physical Activity Lose Weight Lower LDL Healthy Heart # meals w/ low sat. fat # mins vig. activity/wk Healthy Recipes Sneakers Gym Member ship = -1 lb 15 wks > 90 min. (29%) = 300 (27%)
- 68. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 70. Leadership
- 71. Program Staff
- 75. Inputs Activities Outputs Outcomes Impact Eat less fat Vigorous Physical Activity Lose Weight Lower LDL Healthy Heart Healthy Recipes Sneakers Gym Member ship # mins vig. activity/wk 42 wks > 90 min. (81%) # meals w/ low sat. fat = -9 lbs = 820 (75%)
- 76. Inputs Activities Outputs Outcomes Impact Daily Meds Lose Weight Lower LDL Healthy Heart # days with meds Meds Modify Program
- 77. I. The Why: Importance a. Goals and Needs? III. The How: Questions to Ask IV. Q&A and Discussion II. The What: Definitions b. Program plan? c. How are we doing? d. Data and measures? e. Capacity?
- 79. Thank you! Caroline Taggart, MPH Senior Evaluator caroline@viaeval.com www.viaevaluation.com
- 81. Hewlett Foundation Performing Arts Program Strategic Framework; Moss, 2008.
- 82. Output or Outcome Baseline = number from prior to program (if available) or number from first period of program Target = ideal, based on research and reasonable projections (what is realistic to achieve) Indicator = data source; what you are looking at # meals low sat. fat Baseline = unknown; set in Year 1 Target > 15 meals per week with low saturated fat Indicator = meals < estimated low saturated fat level # minutes of vigorous physical activity Baseline = 0 (I never got my heartrate up!) Target > 90 minutes per week (3 30-minute sessions per week) Indicator = minutes of activity with perceived exertion > 15 Lower LDL Baseline = 185 mg/dL Target < 150 mg/dL (still high, but a realistic reduction for my one-year program) Indicator = lipid profile results of LDL Baselines, Targets (Benchmarks), Indicators
- 83. Outcome: the change or result that is desired or expected Indicator: the information (data) that will be used to measure whether the outcome was achieved Easy example: Outcome = lose weight Indicator = weight in pounds More complicated example: Outcome = reduce problem behaviors Indicator = number of suspensions *Often programs list “Reduce Suspensions” as the desired outcome. The program actually wants to reduce problem behaviors, which will be measured by a reduction (hopefully) in suspensions. Same idea: Outcome = improve academic success Indicator = graduation rate Outcomes vs. Indicators
Editor's Notes
- This goes hand-in-hand with understanding effectiveness. Programs that are highly-effective can be brought to scale and serve more people in a community. Similarly, programs that aren’t working can be scaled-back or change course to become more effective. It’s in everyone’s best interest to have effective programs serving as many people as possible in the community.
- RR – started in 1984 with vision to improve reading skills among lowest performers. Engaged in extensive process and outcome evaluations – reformulated curricula, added programs, adjusted, until strong evidence according to WWC. Awarded an I3 in 2010 – first round. Received $45.6 mil to train almost 4k teachers.
- What are the overarching goals of the org? How do those translate into programmatic goals?
- Internal or external; experience in data collection and analysis. Can help on both sides – internal person would be especially useful with multiple programs and on both sides. External person could help build internal capacity before an internal person can be hired; also needed for certain grants when external required.