Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Charity Navigator 2.0 Case Study Presentation


Published on

A case study and discussion of Charity Navigator’s current efforts to move
beyond rating financial accountability to measuring outcomes

  • Be the first to comment

Charity Navigator 2.0 Case Study Presentation

  1. 1. Charity Navigator CN 2.0 Case Study Presentation at Managing to Outcomes ForumPaul Brest, President, The William and Flora Hewlett Foundation With Ken Berger, President & CEO, Charity Navigator, David Bonbright, CEO, Keystone Accountability, Xandy Brown, Pilot Project Coordinator, Charity Navigator, And Professor David Campbell, SUNY Binghamton June 13, 2011
  2. 2. The Data Proves Impact Estimated 3.3 million distinct visitors per year (~5 million hits) 92% say evaluations affected their decision to support individual public charities CN ratings influence decisions on billions in donations annually
  3. 3. CN 2.0Financial Accountability/ Results MoreHealth Transparency Comprehensive Rating System
  4. 4. Note: Strategic Plan is in draft and not yet finalized.
  5. 5. 10,000 CHARITIES RATED BY VOLUNTEER RATERS •  Why 10,000? Because ~70% of all revenues that come into the NP sector annually go to them. •  How to scale this effort? A new process to move beyond our staff and involve consumers in the rating of charities in cause areas they’re passionate about. •  now… 2.0…
  6. 6. Reporting Results R&D Process
  7. 7. What is the charity’s commitment to reporting results?•  Clear commitment to reporting results stated.•  Specified time period for which results are presented.•  Some mission-related results in current period compared to earlier period.•  Reporting distinguishes between activities, outputs and outcomes.•  Credible intention to validate results evidence.
  8. 8. How does the charity demonstrate the demand for its services?•  Reports indicating the aggregate numbers of individuals accessing the charity’s outputs in a given period.•  Evidence of demand for more than half of the charity’s outputs is provided.•  A statement of evidenced demand set against the larger demand that the charity does not presently have the capacity to reach.
  9. 9. Does the charity report its activities and outputs?•  A clear description of the majority of the charity’s mission-related activities and outputs for a defined reporting period•  A significant portion of mission-related activities and outputs reported are compared to an earlier reporting period•  Activities and outputs reported can readily be related to reported expenditure
  10. 10. Does the charity report its outcomes (medium and longer-term results)?•  A clear description of mission-related outcomes achieved by the charity in the reporting period.•  Outcomes are described for more than half of the charity’s mission-related activities or those mission-related outcomes described relate to activities that consume more than half of total charity’s expenditure.
  11. 11. What is the quality of evidence for reported results?•  Evidence of most mission-related outputs includes some element of independent validation•  Low level of outcome evidence•  Medium level of outcome evidence•  High level of outcome evidence: beneficiary feedback•  High level of outcome evidence: independent validation
  12. 12. Does the charity adjust and improve in light of its results?•  Evidence that the organization assesses its mission- related performance in light of what was planned for the reporting period•  Admits mistakes and publicizes corrective actions•  Admits mistakes, publicizes corrective actions and commits itself to validating proposed corrective measures through dialogue with those affected•  Admits mistakes, publicizes corrective actions, and provides evidence that it has in fact validated proposed corrective measures through dialogue with those affected
  13. 13. Framing Questions•  Given the low level of current reporting of results, what are the best initial rating criteria?•  What is the best way to communicate and implement rating criteria that will ratchet up over time?•  What initial criteria will be (a) possible to rate reliably by volunteers and (b) sufficiently achievable to have a basic level of compliance within 12 months, but (c) not be so easy as to be credible and widely gamed.•  Who else needs to be on the bus?
  14. 14. What do student raters learn?•  More questions raised than answered: –  How to differentiate activities, outputs, and outcomes when they are not explicitly labeled? –  How does a charity report results at this level and still have a website that is accessible and clear to lay people? –  How to decrease subjectivity?
  15. 15. How was student feedback•  Materials incorporated? –  Online platform –  Centralized communication•  Rating –  Illustrative examples of where to find information –  4 level confidence scale•  Process –  Thanking volunteers –  Providing background
  16. 16. Binghamton University Courses•  Two courses –  Public Administration •  16 Students •  Issues in Nonprofit Administration –  Social Work •  Advanced Social Work Practice with Communities •  19 Students•  Unique Features –  Local Organization Assessment –  $3,000 in “Philanthropy Incubator” grants –  Student Blog (“Navigating Southern Tier Charity”)
  17. 17. Course Structure•  Public Administration –  Course Focus: Effectiveness and Accountability –  CN Recommended Readings, Forces for Good•  Social Work –  Forces for Good•  Inter-disciplinary Dimensions –  Charity Navigator Assessment Teams –  Local Organization Assessment Teams –  Class Discussion
  18. 18. CN: Students’ Practical Issues•  Assessing Confidence Levels•  Positive Response to 5 Minute Rule•  Most Diligent: Assessments Time Consuming•  Importance of Preparation, Knowledge (SW vs. PA)•  Technology Challenges (CN Responsiveness)
  19. 19. CN: Students’ Philosophical Issues•  Standards Incomplete –  Mistrust –  “Stars” system simplistic. –  Good organizations negatively affected.•  Resource Limitations Constrain Performance•  Assessing Transparency, not Performance –  Performance data depend on trust (not verifiable).•  Discomfort with “Admitting Mistakes”•  Donor vs. Beneficiary Focus
  20. 20. Standards and Students’ Funding Decisions•  Real-world Challenge of Assessment•  Charity Navigator Standards OK After All… –  Financial Measures –  Transparency•  Role of Executive Compensation•  Frustrations with Limitations of Secondary Data –  Assessment Incomplete•  Creation of New Standards –  Role of Social Media –  Student Concerns (small vs. large, effect of contribution)
  21. 21. End of Semester CN Rating Attitudes•  The perfect not the enemy of the good.•  Value of standards•  Not all performance standards apply.•  Create standards for stakeholder dialogue•  SW students: Limited utility for beneficiaries
  22. 22. End Semester Performance Assessment Attitudes•  Effectiveness a social construct•  Negotiate standards with stakeholders.•  More questions than answers; “it depends”•  Importance of responsiveness to performance perceptions.