Successfully reported this slideshow.

Pittsburgh Nonprofit Summit - Measuring Change - Moving From Outputs to Outcomes to Impact

930 views

Published on

More and more funders are only supporting evidence based models. What does this mean and how can your organization implement an evidence based model? A panel of experts has been assembled to answer any and all questions your organization has on how it can move from outputs to outcomes to impact. Brainstorm real strategies that your organization can implement and meet other organizations who are also moving towards an evidence based model.

Published in: Health & Medicine, Business
  • Be the first to comment

Pittsburgh Nonprofit Summit - Measuring Change - Moving From Outputs to Outcomes to Impact

  1. 1. Measuring Change<br />Moving from Outputs to Outcomes to Impact<br />10:00am – 11:15am<br />Facilitator: Lisa Kuzma, Richard King Mellon Foundation<br />Speakers: Mike Bangser, MDRC / Junlei Li, Office of Child Development / Art Maxwell and Pam Meadowcroft, Meadowcroft & Associates / Tim Weidemann, Rondout Consulting <br />
  2. 2. Share your thoughts!<br />Text “Summit” to 57682 with your <br />feedback and thoughts on today’s <br />event!<br />Share your thoughts on Twitter <br />by adding #GPNPSummit to your <br />tweets!<br />
  3. 3. Measuring Change – Moving From Outputs to Outcomes to Impacts<br />
  4. 4. The Context<br />[We are] trying to put much more emphasis on evidence-based policy decisions here at OMB. Wherever possible, we should design new initiatives to build rigorous data about what works and then act on evidence that emerges — expanding the approaches that work best, fine tuning the ones that get mixed results, and shutting down those that are failing. <br /> -- Peter Orszag, former Director of the Office of Management and Budget<br />4<br />
  5. 5. Key Assumptions<br />Agreement on what it means to be “effective”<br />Reliable evidence on which programs are effective, using the proper measures<br />Reliable evidence on the core elements that contribute to effectiveness<br />Documentation and communication of these elements<br />Understanding of how the elements can or should be adapted to local conditions<br />Providers have sufficient resources and capacity to replicate evidence-based practices<br />5<br />
  6. 6. Key Distinctions<br />Outputs (typically, units of service)<br />Gross or absolute outcomes(e.g., employment, learning, or behavioral gains for those who are served)<br />Net impacts(how the outcomes differ from what would have occurred without the intervention)<br />6<br />
  7. 7. 5/11/2011<br />70<br />80<br />40<br />20<br />Job Program Group<br />Control Group<br />Outcomes vs. Impacts: Which program is more effective?<br />100<br />100<br />90<br />90<br />80<br />80<br />70<br />70<br />60<br />60<br /> Percent Placed In A Job <br /> Percent Placed In A Job <br />50<br />50<br />40<br />40<br />30<br />30<br />20<br />20<br />10<br />10<br />0<br />0<br />A<br />B<br />7<br />
  8. 8. Positive Outcomes, But for Whom? <br />
  9. 9. Key Challenges in Replication<br />Rigorous evidence of what works<br />The right level of model specification <br />Proper “fit” between the replicated program model and the local population being served<br />Other factors that may affect results (staffing, funding levels, etc.)<br />Finding useful “leading indicators” for program staff<br />Data issues<br />
  10. 10. Implications for Planning and Practice<br />Need a realistic meeting of the minds on the roles of policymakers, funders, researchers , and providers<br />The goal is fidelity to the right things, at scale and at reasonable cost<br />Need appropriate support for replication and organizational infrastructure (dissemination of research findings , tools and training, data systems, etc.)<br />Beware of potential distortions in interpreting outcomes – especially the influence of participant selection<br />Recognize that moving from outputs to outcomes to impacts is an evolutionary process<br />
  11. 11. For More Information<br />Michael Bangser<br />mike.bbangser@mdrc.org<br /> 860-586-8577<br />www.mdrc.org<br />
  12. 12. Thank you from the Greater Pittsburgh Nonprofit Partnership!<br />
  13. 13. Workshop Evaluation<br />Text “Impact” to 57682 with your answers to the following questions:<br />Please rate the overall value of this workshop<br /> E – Excellent G – Good P – Poor F – Fair <br />Did you learn anything that you will apply at your own organization?<br /> Y – Yes N - No<br />Please text other comments and feedback.<br />Submit by hitting “send!” You will receive an auto-reply from the GPNP.<br />Sample text: “Impact E Y I’ve learned great tips today.”<br />

×