The document discusses the history and future of performance metrics and evaluation in large foundations. It outlines how evaluation has become ubiquitous in foundations since the 1960s-1970s, driven by laws, associations, and publications. It also describes different methodologies used for evaluation and examples of social metrics and performance metrics developed by some foundations to measure outcomes. Going forward, it suggests performance and social metrics are unlikely to dominate due to foundations' external focus on building field capacity rather than internal decision-making, and that foundations will continue seeking legitimacy through diverse evaluation approaches.

1. Performance Metrics and Evaluation in Large Foundations Victor Kuo, PhD Bill & Melinda Gates Foundation November 10, 2007

2. Presentation Outline Historical Overview Methodological Overview Performance Metrics in Foundations Considerations for the Future Future Performance Metrics Methodology History

3. Historical Overview Dobkin Hall, 2004 In Braverman et al. Fleishman, 2007 Frumkin, 2006 Evaluation has become ubiquitous in foundations. Future Performance Metrics Methodology History

4. Historical Overview Evaluation’s presence in philanthropy has a history marked by laws, formalized associations, and publications. Where do Performance Metrics fall? RESEARCH RESPONSIBILITY RETURNS 1996: Measuring Program Outcomes: A Practical Approach United Way 1993: Government Performance & Results Act 1998: Evaluation Handbook Kellogg Foundation 2002: Grantmakers for Effective Organizations 1973: Brim’s essay on evaluation Russell Sage Foundation 1969: Tax Reform Act 1986: American Evaluation Association 1974: MDRC Ford Foundation 2007: Critical Impact Awards Council of Foundations 1997: Performance Measurement in Public & Nonprofits New Directions for Evaluation AEA Future Performance Metrics Methodology History

5. Methodological Overview Evaluation methodologies also are varied. Where do performance metrics and social indicators fall? (Brown & Corbett, 1997) Future Performance Metrics Methodology History Monitoring Theory-Driven Quasi-Experiments Experiments

6. Methodological Overview Causal Descriptive Sampling Census Knowledge Improvement/ Formative Judgment/ Summative Future Performance Metrics Methodology History

7. Social Metrics There are some good examples of social metrics developed and used by foundations. Future Performance Metrics Methodology History Kids Count - 1990 Annie E. Casey Foundation Community Indicators Project - 2001 John S. and James L. Knight Foundation

8. Performance Metrics External vs. Internal Focus Business & Management Balanced Scorecard (Kaplan & Norton, 1992) Financial Customer satisfaction Internal business Innovation and learning Future Performance Metrics Methodology History

9. Performance Metrics Dashboard Gates Education Program There are some emerging examples of performance metrics being developed by some foundations. Impact Framework Robert Wood Johnson Future Performance Metrics Methodology History Grantee Perception Report Center for Effective Philanthropy

10. Considerations for the Future Will Performance and Social Metrics Dominate? Not likely (my opinion!) Doesn’t make decision-making easier (Platntz, Greenway, Hendricks, NDE 1997) Internal focus goes against instinct and best-practices of building external capacity in the field and among grantees. Trends 70’s: social science research 90’s: accountability 00’s: managerial and organizational effectiveness, professionalism 10’s: ??? Future Performance Metrics Methodology History

11. Considerations for the Future Foundations Will Continue to Seek Legitimacy Foundations have no mandate to evaluate, or to evaluate in a certain way They will respond to environmental pressures and to one another (Powell & DiMaggio, 1983). They seek innovation in programs and in operations Evaluators Working With Foundations Should Be Comfortable with Diverse Analytic Approaches Diagnose and adapt to changing internal & external needs Philanthropy will draw from a wide-range of evaluation professionals Future Performance Metrics Methodology History

12. Considerations for the Future What is “Impact”? (Fleishman, 2007) Major benefits to the public 911, Robert Wood Johnson Expansion of knowledge Green Revolution, food production, Rockefeller Movements Civil Rights litigation, Ford Scaling a Model Micro-finance, micro-loans, Ford and others Physical Buildings Libraries, Carnegie Will Performance and Social Metrics get us to impact? Future Performance Metrics Methodology History