Performance Metrics and Evaluation in Large Foundations Victor Kuo, PhD Bill & Melinda Gates Foundation November 10, 2007

Presentation Outline Historical Overview Methodological Overview Performance Metrics in Foundations Considerations for the Future Future Performance Metrics Methodology History

Historical Overview

Methodological Overview

Performance Metrics in Foundations

Considerations for the Future

Historical Overview Dobkin Hall, 2004 In Braverman et al. Fleishman, 2007 Frumkin, 2006 Evaluation has become ubiquitous in foundations. Future Performance Metrics Methodology History

Historical Overview Evaluation’s presence in philanthropy has a history marked by laws, formalized associations, and publications. Where do Performance Metrics fall? RESEARCH RESPONSIBILITY RETURNS 1996: Measuring Program Outcomes: A Practical Approach United Way 1993: Government Performance & Results Act 1998: Evaluation Handbook Kellogg Foundation 2002: Grantmakers for Effective Organizations 1973: Brim’s essay on evaluation Russell Sage Foundation 1969: Tax Reform Act 1986: American Evaluation Association 1974: MDRC Ford Foundation 2007: Critical Impact Awards Council of Foundations 1997: Performance Measurement in Public & Nonprofits New Directions for Evaluation AEA Future Performance Metrics Methodology History

Methodological Overview Evaluation methodologies also are varied. Where do performance metrics and social indicators fall? (Brown & Corbett, 1997) Future Performance Metrics Methodology History Monitoring Theory-Driven Quasi-Experiments Experiments

Methodological Overview Causal Descriptive Sampling Census Knowledge Improvement/ Formative Judgment/ Summative Future Performance Metrics Methodology History

Social Metrics There are some good examples of social metrics developed and used by foundations. Future Performance Metrics Methodology History Kids Count - 1990 Annie E. Casey Foundation Community Indicators Project - 2001 John S. and James L. Knight Foundation

Performance Metrics External vs. Internal Focus Business & Management Balanced Scorecard (Kaplan & Norton, 1992) Financial Customer satisfaction Internal business Innovation and learning Future Performance Metrics Methodology History

External vs. Internal Focus

Business & Management

Balanced Scorecard (Kaplan & Norton, 1992)

Financial

Customer satisfaction

Internal business

Innovation and learning

Performance Metrics Dashboard Gates Education Program There are some emerging examples of performance metrics being developed by some foundations. Impact Framework Robert Wood Johnson Future Performance Metrics Methodology History Grantee Perception Report Center for Effective Philanthropy

Considerations for the Future Will Performance and Social Metrics Dominate? Not likely (my opinion!) Doesn’t make decision-making easier (Platntz, Greenway, Hendricks, NDE 1997) Internal focus goes against instinct and best-practices of building external capacity in the field and among grantees. Trends 70’s: social science research 90’s: accountability 00’s: managerial and organizational effectiveness, professionalism 10’s: ??? Future Performance Metrics Methodology History

Will Performance and Social Metrics Dominate?

Not likely (my opinion!)

Doesn’t make decision-making easier (Platntz, Greenway, Hendricks, NDE 1997)

Internal focus goes against instinct and best-practices of building external capacity in the field and among grantees.

Trends

70’s: social science research

90’s: accountability

00’s: managerial and organizational effectiveness, professionalism

10’s: ???

Considerations for the Future Foundations Will Continue to Seek Legitimacy Foundations have no mandate to evaluate, or to evaluate in a certain way They will respond to environmental pressures and to one another (Powell & DiMaggio, 1983). They seek innovation in programs and in operations Evaluators Working With Foundations Should Be Comfortable with Diverse Analytic Approaches Diagnose and adapt to changing internal & external needs Philanthropy will draw from a wide-range of evaluation professionals Future Performance Metrics Methodology History

Foundations Will Continue to Seek Legitimacy

Foundations have no mandate to evaluate, or to evaluate in a certain way

They will respond to environmental pressures and to one another (Powell & DiMaggio, 1983).

They seek innovation in programs and in operations

Evaluators Working With Foundations Should Be Comfortable with Diverse Analytic Approaches

Diagnose and adapt to changing internal & external needs

Philanthropy will draw from a wide-range of evaluation professionals

Considerations for the Future What is “Impact”? (Fleishman, 2007) Major benefits to the public 911, Robert Wood Johnson Expansion of knowledge Green Revolution, food production, Rockefeller Movements Civil Rights litigation, Ford Scaling a Model Micro-finance, micro-loans, Ford and others Physical Buildings Libraries, Carnegie Will Performance and Social Metrics get us to impact? Future Performance Metrics Methodology History

What is “Impact”? (Fleishman, 2007)

Major benefits to the public

911, Robert Wood Johnson

Expansion of knowledge

Green Revolution, food production, Rockefeller

Movements

Civil Rights litigation, Ford

Scaling a Model

Micro-finance, micro-loans, Ford and others

Physical Buildings

Libraries, Carnegie

Will Performance and Social Metrics get us to impact?