Program Evaluation:Moving from Evaluation to Learning
2Session Agenda• Introductions• The Need for Learning• Why Nonprofits Aren’t Learning• Evaluation vs. Evaluative Learning•...
The Need for Learning
4Leadership = SuccessLeadership is key to:GrowthSustainabilityLifecycle AdvancementEffectiveness
5There’s More to Leadership…• Advancing effective communication of mission and vision tointernal and external stakeholders...
6The Key to Leading Is LearningOnly one in four nonprofit organizations is an effective learner…The following organization...
Why Aren’t Nonprofits Learning?
8The Investment Model Is All Wrong• There’s no “outcome synchronicity” between theinvestor and investee• Effectiveness and...
9What Results Do Investors, Business Leaders, & Consumers Want?Example: In-Home Support Services to New Parents, For-Profi...
Why Evaluation Doesn’t AlwaysLead to Learning
11Why Evaluation Isn’t Achieving Learning1. Makes uncontrollable community impact indicators themetric of accountability2....
Evaluation vs. Evaluative Learning
13The Difference Between Evaluation and EvaluativeLearningEvaluation: the owner determines success by overallwins, losses ...
14ResourcesWholeProgramShort-TermOutcomesLong-TermOutcomesCommunity/ SocialImpactEvaluationDetermining a “Whole” Program’s...
15ProgramResourcesProgramElementsAchievableOutcomes/Next ActionsStudiessay…Community /Social ImpactEvaluative LearningMeas...
16Six Learning Practices that Facilitate SustainableGrowthOrganizations are significantly more likely to sustain and grow ...
17Evaluative Learning Facilitates Sustainable Growth41% of nonprofitsgrow faster thanthe rate of inflationover a three yea...
18Guiding Steps for Conducting Evaluative Learning1. Measure achievable outcomes, next actions, direct results.2. Gather l...
19Step One: Identify Your Achievable Outcomes• It’s all about the path to behavior change:– Awareness +– Knowledge +– Atti...
20Evaluative Learning Processes to Try1. Positive Deviant Experiment2. Literature Review to Make the Case for Proximate Ou...
Thanks!
Upcoming SlideShare
Loading in …5
×

Moving from evaluation to learning peter york

566 views

Published on

Moving from Evaluation to Learning by Peter York

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
566
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Moving from evaluation to learning peter york

  1. 1. Program Evaluation:Moving from Evaluation to Learning
  2. 2. 2Session Agenda• Introductions• The Need for Learning• Why Nonprofits Aren’t Learning• Evaluation vs. Evaluative Learning• Nonprofit Evaluative Learning – Steps, Processes & Tools
  3. 3. The Need for Learning
  4. 4. 4Leadership = SuccessLeadership is key to:GrowthSustainabilityLifecycle AdvancementEffectiveness
  5. 5. 5There’s More to Leadership…• Advancing effective communication of mission and vision tointernal and external stakeholders• Engaging internal and external stakeholders in planning• Taking decisive action when faced with challenges• Making decisions anchored in cost-effectiveness• Demanding accountability that includes demonstratedsuccess with those being served/targetedOnly one in four nonprofit organizations is well led…
  6. 6. 6The Key to Leading Is LearningOnly one in four nonprofit organizations is an effective learner…The following organizational learning behaviors are significantly andsingly the biggest predictor of organizational leadership:• Creating sophisticated financial, operational, programmatic andenvironmental data-gathering and learning processes• Spending time leveraging program data for making meaning,decisions, designs and plans, not accountability or validation• Infusing learning into ALL planning, anchored in program success• Taking immediate and decisive action, particularly at a humanresource level, as indicated by strategy research findings
  7. 7. Why Aren’t Nonprofits Learning?
  8. 8. 8The Investment Model Is All Wrong• There’s no “outcome synchronicity” between theinvestor and investee• Effectiveness and accountability must be viewedthrough the measurement of proximate effect• Proximate cause-and-effect is the only way!• Learning requires understanding the cause, not theeffect• There’s no Research & Development forprograms/initiatives
  9. 9. 9What Results Do Investors, Business Leaders, & Consumers Want?Example: In-Home Support Services to New Parents, For-Profits vs. NonprofitsInvestors(Funders)BusinessLeadersConsumersFor-Profit Results:Just Give Me (or Show Me) the DirectResults, Please…Nonprofit Accountability:Just Give Me the Direct Results, But Somehow Prove to“Them” That We Can Do Much More…Recuperation, parent-child bonding, healthyadjustment to family change, tools for care andfeeding, stress-reducing routines and habits, bettercommunication skills with providers• Same Direct Resultsas the For-ProfitBusiness• Child Development, SchoolReadiness, CrimeReduction• Parental Reduction in ChildAbuse & Neglect, MaternalHealthNonprofitBusinessLeaders &ClientsFunders(Investors)
  10. 10. Why Evaluation Doesn’t AlwaysLead to Learning
  11. 11. 11Why Evaluation Isn’t Achieving Learning1. Makes uncontrollable community impact indicators themetric of accountability2. Evaluates the “whole” program/strategic effort to determineif it worked, rather than the combinations of programcomponents to determine how it works3. Aspires to a scientific research design ideal that seekswidespread generalizability, rather than contextualapplications4. Gathers data from the wrong source
  12. 12. Evaluation vs. Evaluative Learning
  13. 13. 13The Difference Between Evaluation and EvaluativeLearningEvaluation: the owner determines success by overallwins, losses and trophies.Evaluative Learning: the coach/manager figures out howto improve the odds of success for the next game byanalyzing the play of the previous ones.
  14. 14. 14ResourcesWholeProgramShort-TermOutcomesLong-TermOutcomesCommunity/ SocialImpactEvaluationDetermining a “Whole” Program’s Long-Term Impact to Judge Social Value
  15. 15. 15ProgramResourcesProgramElementsAchievableOutcomes/Next ActionsStudiessay…Community /Social ImpactEvaluative LearningMeasuring Program Success for “Best Practice” Improvement
  16. 16. 16Six Learning Practices that Facilitate SustainableGrowthOrganizations are significantly more likely to sustain and grow if theyengage in the following evaluative learning behaviors: NOT evaluating to decide if a whole program works; rather, figuring outwhen and how it works. Gathering data from recipients. Determining outcomes by listening to recipient changes and nextactions. Engaging people in making meaning from data. Developing performance management processes, tools and systems. Prioritizing cost-per-outcome.
  17. 17. 17Evaluative Learning Facilitates Sustainable Growth41% of nonprofitsgrow faster thanthe rate of inflationover a three yearperiodNonprofits whereleaders engage inthese behaviorsare 2.5 timesmore likely togrow faster thanthe rate ofinflation.7%5%4%2%Conducting AllR&D Behaviors(0% of all NPs)ConductingMany R&DBehaviors (5%)ConductingSome R&DBehaviors(45%)ConducingVery Few R&DBehaviors(25%)Conducting NOR&D Behaviors(25%)AverageAnnualGrowthRateAverage Annual Growth Rate(Mean Per Year, Based on Three Consecutive Years of Data)
  18. 18. 18Guiding Steps for Conducting Evaluative Learning1. Measure achievable outcomes, next actions, direct results.2. Gather lots of data QUICKLY, by quantifying the qualitative.3. Ask the participant/recipient.4. Analyze data for patterns of cause-and-effect.5. Gather a few key illustrative stories.6. Spend time making meaning and making improvements.
  19. 19. 19Step One: Identify Your Achievable Outcomes• It’s all about the path to behavior change:– Awareness +– Knowledge +– Attitude +– Motivation +– Skills +– Opportunity +– Behaviors =___________________Habits
  20. 20. 20Evaluative Learning Processes to Try1. Positive Deviant Experiment2. Literature Review to Make the Case for Proximate Outcomes3. Proximate Logic Model Creation4. Survey Data Project5. Qualitative Data Project6. Creation of Ongoing R&D and Performance Management Tools
  21. 21. Thanks!

×