Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moving from evaluation to learning peter york


Published on

Moving from Evaluation to Learning by Peter York

  • Be the first to comment

  • Be the first to like this

Moving from evaluation to learning peter york

  1. 1. Program Evaluation:Moving from Evaluation to Learning
  2. 2. 2Session Agenda• Introductions• The Need for Learning• Why Nonprofits Aren’t Learning• Evaluation vs. Evaluative Learning• Nonprofit Evaluative Learning – Steps, Processes & Tools
  3. 3. The Need for Learning
  4. 4. 4Leadership = SuccessLeadership is key to:GrowthSustainabilityLifecycle AdvancementEffectiveness
  5. 5. 5There’s More to Leadership…• Advancing effective communication of mission and vision tointernal and external stakeholders• Engaging internal and external stakeholders in planning• Taking decisive action when faced with challenges• Making decisions anchored in cost-effectiveness• Demanding accountability that includes demonstratedsuccess with those being served/targetedOnly one in four nonprofit organizations is well led…
  6. 6. 6The Key to Leading Is LearningOnly one in four nonprofit organizations is an effective learner…The following organizational learning behaviors are significantly andsingly the biggest predictor of organizational leadership:• Creating sophisticated financial, operational, programmatic andenvironmental data-gathering and learning processes• Spending time leveraging program data for making meaning,decisions, designs and plans, not accountability or validation• Infusing learning into ALL planning, anchored in program success• Taking immediate and decisive action, particularly at a humanresource level, as indicated by strategy research findings
  7. 7. Why Aren’t Nonprofits Learning?
  8. 8. 8The Investment Model Is All Wrong• There’s no “outcome synchronicity” between theinvestor and investee• Effectiveness and accountability must be viewedthrough the measurement of proximate effect• Proximate cause-and-effect is the only way!• Learning requires understanding the cause, not theeffect• There’s no Research & Development forprograms/initiatives
  9. 9. 9What Results Do Investors, Business Leaders, & Consumers Want?Example: In-Home Support Services to New Parents, For-Profits vs. NonprofitsInvestors(Funders)BusinessLeadersConsumersFor-Profit Results:Just Give Me (or Show Me) the DirectResults, Please…Nonprofit Accountability:Just Give Me the Direct Results, But Somehow Prove to“Them” That We Can Do Much More…Recuperation, parent-child bonding, healthyadjustment to family change, tools for care andfeeding, stress-reducing routines and habits, bettercommunication skills with providers• Same Direct Resultsas the For-ProfitBusiness• Child Development, SchoolReadiness, CrimeReduction• Parental Reduction in ChildAbuse & Neglect, MaternalHealthNonprofitBusinessLeaders &ClientsFunders(Investors)
  10. 10. Why Evaluation Doesn’t AlwaysLead to Learning
  11. 11. 11Why Evaluation Isn’t Achieving Learning1. Makes uncontrollable community impact indicators themetric of accountability2. Evaluates the “whole” program/strategic effort to determineif it worked, rather than the combinations of programcomponents to determine how it works3. Aspires to a scientific research design ideal that seekswidespread generalizability, rather than contextualapplications4. Gathers data from the wrong source
  12. 12. Evaluation vs. Evaluative Learning
  13. 13. 13The Difference Between Evaluation and EvaluativeLearningEvaluation: the owner determines success by overallwins, losses and trophies.Evaluative Learning: the coach/manager figures out howto improve the odds of success for the next game byanalyzing the play of the previous ones.
  14. 14. 14ResourcesWholeProgramShort-TermOutcomesLong-TermOutcomesCommunity/ SocialImpactEvaluationDetermining a “Whole” Program’s Long-Term Impact to Judge Social Value
  15. 15. 15ProgramResourcesProgramElementsAchievableOutcomes/Next ActionsStudiessay…Community /Social ImpactEvaluative LearningMeasuring Program Success for “Best Practice” Improvement
  16. 16. 16Six Learning Practices that Facilitate SustainableGrowthOrganizations are significantly more likely to sustain and grow if theyengage in the following evaluative learning behaviors: NOT evaluating to decide if a whole program works; rather, figuring outwhen and how it works. Gathering data from recipients. Determining outcomes by listening to recipient changes and nextactions. Engaging people in making meaning from data. Developing performance management processes, tools and systems. Prioritizing cost-per-outcome.
  17. 17. 17Evaluative Learning Facilitates Sustainable Growth41% of nonprofitsgrow faster thanthe rate of inflationover a three yearperiodNonprofits whereleaders engage inthese behaviorsare 2.5 timesmore likely togrow faster thanthe rate ofinflation.7%5%4%2%Conducting AllR&D Behaviors(0% of all NPs)ConductingMany R&DBehaviors (5%)ConductingSome R&DBehaviors(45%)ConducingVery Few R&DBehaviors(25%)Conducting NOR&D Behaviors(25%)AverageAnnualGrowthRateAverage Annual Growth Rate(Mean Per Year, Based on Three Consecutive Years of Data)
  18. 18. 18Guiding Steps for Conducting Evaluative Learning1. Measure achievable outcomes, next actions, direct results.2. Gather lots of data QUICKLY, by quantifying the qualitative.3. Ask the participant/recipient.4. Analyze data for patterns of cause-and-effect.5. Gather a few key illustrative stories.6. Spend time making meaning and making improvements.
  19. 19. 19Step One: Identify Your Achievable Outcomes• It’s all about the path to behavior change:– Awareness +– Knowledge +– Attitude +– Motivation +– Skills +– Opportunity +– Behaviors =___________________Habits
  20. 20. 20Evaluative Learning Processes to Try1. Positive Deviant Experiment2. Literature Review to Make the Case for Proximate Outcomes3. Proximate Logic Model Creation4. Survey Data Project5. Qualitative Data Project6. Creation of Ongoing R&D and Performance Management Tools
  21. 21. Thanks!