AEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations


Published on

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • NOTES FOR USAGE Help for presentation creation and template usage can be found in FND_TemplateAddendum.ppt. Be sure to first Save As this file with another name. (By renaming the file, you preserve the template for the next person.) TITLE SLIDE: Background photo can be deleted or replaced. For additional photos, see PHOTO LIBRARY.
  • AEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations

    1. 1. Performance Metrics and Evaluation in Large Foundations Victor Kuo, PhD Bill & Melinda Gates Foundation November 10, 2007
    2. 2. Presentation Outline <ul><li>Historical Overview </li></ul><ul><li>Methodological Overview </li></ul><ul><li>Performance Metrics in Foundations </li></ul><ul><li>Considerations for the Future </li></ul>Future Performance Metrics Methodology History
    3. 3. Historical Overview Dobkin Hall, 2004 In Braverman et al. Fleishman, 2007 Frumkin, 2006 Evaluation has become ubiquitous in foundations. Future Performance Metrics Methodology History
    4. 4. Historical Overview Evaluation’s presence in philanthropy has a history marked by laws, formalized associations, and publications. Where do Performance Metrics fall? RESEARCH RESPONSIBILITY RETURNS 1996: Measuring Program Outcomes: A Practical Approach United Way 1993: Government Performance & Results Act 1998: Evaluation Handbook Kellogg Foundation 2002: Grantmakers for Effective Organizations 1973: Brim’s essay on evaluation Russell Sage Foundation 1969: Tax Reform Act 1986: American Evaluation Association 1974: MDRC Ford Foundation 2007: Critical Impact Awards Council of Foundations 1997: Performance Measurement in Public & Nonprofits New Directions for Evaluation AEA Future Performance Metrics Methodology History
    5. 5. Methodological Overview Evaluation methodologies also are varied. Where do performance metrics and social indicators fall? (Brown & Corbett, 1997) Future Performance Metrics Methodology History Monitoring Theory-Driven Quasi-Experiments Experiments
    6. 6. Methodological Overview Causal Descriptive Sampling Census Knowledge Improvement/ Formative Judgment/ Summative Future Performance Metrics Methodology History
    7. 7. Social Metrics There are some good examples of social metrics developed and used by foundations. Future Performance Metrics Methodology History Kids Count - 1990 Annie E. Casey Foundation Community Indicators Project - 2001 John S. and James L. Knight Foundation
    8. 8. Performance Metrics <ul><li>External vs. Internal Focus </li></ul><ul><li>Business & Management </li></ul><ul><ul><li>Balanced Scorecard (Kaplan & Norton, 1992) </li></ul></ul><ul><ul><ul><li>Financial </li></ul></ul></ul><ul><ul><ul><li>Customer satisfaction </li></ul></ul></ul><ul><ul><ul><li>Internal business </li></ul></ul></ul><ul><ul><ul><li>Innovation and learning </li></ul></ul></ul>Future Performance Metrics Methodology History
    9. 9. Performance Metrics Dashboard Gates Education Program There are some emerging examples of performance metrics being developed by some foundations. Impact Framework Robert Wood Johnson Future Performance Metrics Methodology History Grantee Perception Report Center for Effective Philanthropy
    10. 10. Considerations for the Future <ul><li>Will Performance and Social Metrics Dominate? </li></ul><ul><ul><li>Not likely (my opinion!) </li></ul></ul><ul><ul><li>Doesn’t make decision-making easier (Platntz, Greenway, Hendricks, NDE 1997) </li></ul></ul><ul><ul><li>Internal focus goes against instinct and best-practices of building external capacity in the field and among grantees. </li></ul></ul><ul><li>Trends </li></ul><ul><ul><li>70’s: social science research </li></ul></ul><ul><ul><li>90’s: accountability </li></ul></ul><ul><ul><li>00’s: managerial and organizational effectiveness, professionalism </li></ul></ul><ul><ul><li>10’s: ??? </li></ul></ul>Future Performance Metrics Methodology History
    11. 11. Considerations for the Future <ul><li>Foundations Will Continue to Seek Legitimacy </li></ul><ul><ul><li>Foundations have no mandate to evaluate, or to evaluate in a certain way </li></ul></ul><ul><ul><li>They will respond to environmental pressures and to one another (Powell & DiMaggio, 1983). </li></ul></ul><ul><ul><li>They seek innovation in programs and in operations </li></ul></ul><ul><li>Evaluators Working With Foundations Should Be Comfortable with Diverse Analytic Approaches </li></ul><ul><ul><li>Diagnose and adapt to changing internal & external needs </li></ul></ul><ul><ul><li>Philanthropy will draw from a wide-range of evaluation professionals </li></ul></ul>Future Performance Metrics Methodology History
    12. 12. Considerations for the Future <ul><li>What is “Impact”? (Fleishman, 2007) </li></ul><ul><ul><li>Major benefits to the public </li></ul></ul><ul><ul><ul><li>911, Robert Wood Johnson </li></ul></ul></ul><ul><ul><li>Expansion of knowledge </li></ul></ul><ul><ul><ul><li>Green Revolution, food production, Rockefeller </li></ul></ul></ul><ul><ul><li>Movements </li></ul></ul><ul><ul><ul><li>Civil Rights litigation, Ford </li></ul></ul></ul><ul><ul><li>Scaling a Model </li></ul></ul><ul><ul><ul><li>Micro-finance, micro-loans, Ford and others </li></ul></ul></ul><ul><ul><li>Physical Buildings </li></ul></ul><ul><ul><ul><li>Libraries, Carnegie </li></ul></ul></ul><ul><li>Will Performance and Social Metrics get us to impact? </li></ul>Future Performance Metrics Methodology History