Moving from evaluation to learning peter york

  • 173 views
Uploaded on

Moving from Evaluation to Learning by Peter York

Moving from Evaluation to Learning by Peter York

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
173
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Program Evaluation:Moving from Evaluation to Learning
  • 2. 2Session Agenda• Introductions• The Need for Learning• Why Nonprofits Aren’t Learning• Evaluation vs. Evaluative Learning• Nonprofit Evaluative Learning – Steps, Processes & Tools
  • 3. The Need for Learning
  • 4. 4Leadership = SuccessLeadership is key to:GrowthSustainabilityLifecycle AdvancementEffectiveness
  • 5. 5There’s More to Leadership…• Advancing effective communication of mission and vision tointernal and external stakeholders• Engaging internal and external stakeholders in planning• Taking decisive action when faced with challenges• Making decisions anchored in cost-effectiveness• Demanding accountability that includes demonstratedsuccess with those being served/targetedOnly one in four nonprofit organizations is well led…
  • 6. 6The Key to Leading Is LearningOnly one in four nonprofit organizations is an effective learner…The following organizational learning behaviors are significantly andsingly the biggest predictor of organizational leadership:• Creating sophisticated financial, operational, programmatic andenvironmental data-gathering and learning processes• Spending time leveraging program data for making meaning,decisions, designs and plans, not accountability or validation• Infusing learning into ALL planning, anchored in program success• Taking immediate and decisive action, particularly at a humanresource level, as indicated by strategy research findings
  • 7. Why Aren’t Nonprofits Learning?
  • 8. 8The Investment Model Is All Wrong• There’s no “outcome synchronicity” between theinvestor and investee• Effectiveness and accountability must be viewedthrough the measurement of proximate effect• Proximate cause-and-effect is the only way!• Learning requires understanding the cause, not theeffect• There’s no Research & Development forprograms/initiatives
  • 9. 9What Results Do Investors, Business Leaders, & Consumers Want?Example: In-Home Support Services to New Parents, For-Profits vs. NonprofitsInvestors(Funders)BusinessLeadersConsumersFor-Profit Results:Just Give Me (or Show Me) the DirectResults, Please…Nonprofit Accountability:Just Give Me the Direct Results, But Somehow Prove to“Them” That We Can Do Much More…Recuperation, parent-child bonding, healthyadjustment to family change, tools for care andfeeding, stress-reducing routines and habits, bettercommunication skills with providers• Same Direct Resultsas the For-ProfitBusiness• Child Development, SchoolReadiness, CrimeReduction• Parental Reduction in ChildAbuse & Neglect, MaternalHealthNonprofitBusinessLeaders &ClientsFunders(Investors)
  • 10. Why Evaluation Doesn’t AlwaysLead to Learning
  • 11. 11Why Evaluation Isn’t Achieving Learning1. Makes uncontrollable community impact indicators themetric of accountability2. Evaluates the “whole” program/strategic effort to determineif it worked, rather than the combinations of programcomponents to determine how it works3. Aspires to a scientific research design ideal that seekswidespread generalizability, rather than contextualapplications4. Gathers data from the wrong source
  • 12. Evaluation vs. Evaluative Learning
  • 13. 13The Difference Between Evaluation and EvaluativeLearningEvaluation: the owner determines success by overallwins, losses and trophies.Evaluative Learning: the coach/manager figures out howto improve the odds of success for the next game byanalyzing the play of the previous ones.
  • 14. 14ResourcesWholeProgramShort-TermOutcomesLong-TermOutcomesCommunity/ SocialImpactEvaluationDetermining a “Whole” Program’s Long-Term Impact to Judge Social Value
  • 15. 15ProgramResourcesProgramElementsAchievableOutcomes/Next ActionsStudiessay…Community /Social ImpactEvaluative LearningMeasuring Program Success for “Best Practice” Improvement
  • 16. 16Six Learning Practices that Facilitate SustainableGrowthOrganizations are significantly more likely to sustain and grow if theyengage in the following evaluative learning behaviors: NOT evaluating to decide if a whole program works; rather, figuring outwhen and how it works. Gathering data from recipients. Determining outcomes by listening to recipient changes and nextactions. Engaging people in making meaning from data. Developing performance management processes, tools and systems. Prioritizing cost-per-outcome.
  • 17. 17Evaluative Learning Facilitates Sustainable Growth41% of nonprofitsgrow faster thanthe rate of inflationover a three yearperiodNonprofits whereleaders engage inthese behaviorsare 2.5 timesmore likely togrow faster thanthe rate ofinflation.7%5%4%2%Conducting AllR&D Behaviors(0% of all NPs)ConductingMany R&DBehaviors (5%)ConductingSome R&DBehaviors(45%)ConducingVery Few R&DBehaviors(25%)Conducting NOR&D Behaviors(25%)AverageAnnualGrowthRateAverage Annual Growth Rate(Mean Per Year, Based on Three Consecutive Years of Data)
  • 18. 18Guiding Steps for Conducting Evaluative Learning1. Measure achievable outcomes, next actions, direct results.2. Gather lots of data QUICKLY, by quantifying the qualitative.3. Ask the participant/recipient.4. Analyze data for patterns of cause-and-effect.5. Gather a few key illustrative stories.6. Spend time making meaning and making improvements.
  • 19. 19Step One: Identify Your Achievable Outcomes• It’s all about the path to behavior change:– Awareness +– Knowledge +– Attitude +– Motivation +– Skills +– Opportunity +– Behaviors =___________________Habits
  • 20. 20Evaluative Learning Processes to Try1. Positive Deviant Experiment2. Literature Review to Make the Case for Proximate Outcomes3. Proximate Logic Model Creation4. Survey Data Project5. Qualitative Data Project6. Creation of Ongoing R&D and Performance Management Tools
  • 21. Thanks!