In an increasingly challenging donor environment, funders want more meaning reporting of success and outcomes by nonprofits. This webinar provides insights and knowledge that can mean the difference between scaling up - or dialing down - key initiatives.

1. Measuring Success in
Patient Advocacy Initiatives
July 9, 2014

2. Welcome!
Lori Melançon,
Senior Director,
Corporate Affairs,
Onyx Pharmaceuticals
Ken Berger,
President & CEO,
Charity Navigator
Carmen Perez,
Manager, Measurement
and Standards,
CECP

3. Progress in Cancer:
Measuring Success in Patient
Advocacy Initiatives
A Metrics Study
Lori Melançon
Senior Director, Corporate Affairs
Onyx Pharmaceuticals

4. Measuring Impact in the Evolving World
of Nonprofit Organizations (NPOs)
Increased pressure to measure impact
Fierce competition for funding
Measuring success a daunting challenge

5. Progress in Cancer Study Overview
Objectives
• Evaluate how diverse NPOs in cancer measure impact against goals to
assess trends and best practices
Rx4good Methodology
Primary Research: Group Self-Assessment
▪ 24 NPOs participated in self-assessment of metrics best practices via 8-item
Survey Monkey questionnaire
Secondary Research: Assessment of Websites and Annual Reports
▪ 45-question survey evaluated how 35 NPOs report on diverse measures of
progress against goals
1
2

6. Study Sample
Websites and annual reports of 35 NPOs assessed:
Self-assessments completed by 24 (69%) of the 35 organizations

7. Group Demographics
The majority of the 35 assessed organizations have been
operating for 20+ years with annual revenue of $1M–$5M
Age of Organization Annual Revenue
<$1M 20% (n=7)
$1M - $5M 31% (n=11)
$5M – $10M 23% (n=8)
$10M - $30M 11% (n=4)
>$30M 11% (n=4)
N/A 3% (n=1)
20+ Years 49% (n=17)
10-20 Years 43% (n=15)
5-10 Years 6% (n=2)
1-5 Years 2% (n=1)

8. Primary Research Findings
NPO Self-Assessment of Metrics Best Practices

9. Survey Revealed NPO’s Perspective on
Metrics Reporting…
75% of NPOs have
changed their view
about metrics during
last three years
• increased
understanding of
value of metrics
• funder
expectations
96%
Metrics are essential for
evaluating success
against goals
33%
The work we do is
difficult to measure
25%
Metrics are important
but we don’t have the
resources to
measure our
programs
8%
We would rather
invest in
programs than in
measuring results

10. Key Obstacles to Measuring Impact
Lack of Resources Determining Outcomes Change
We have limited time, staff and funding to
do all the things we want to do…And it's hard
to increase staff size without
an increase in funding.
Measuring impact takes so much staff
and time that the programs do not get
the attention they deserve.
It is the wrong way around.
Not all of our programs are easy to
track, especially grassroots and
advocacy initiatives.
It can be can be a challenge to determine
the best way to measure impact. It can also
be challenging to determine if behavior
has really changed due to a program
or an event.
“
“
“ “
“ “
“ “

11. NPOs Measure Impact in Many Ways,
Led by Progress Achieved Against Goals
96%
92%
92%
92%
79%
75%
75%
67%
67%
54%
Progress Achieved Against Goals
Numbers Reached, Attended Events
Actions Taken by Stakeholders as a Result of
Programs
Social Media or Web Engagement
Media Coverage
Funding Received
Perceptions Changed or
Reinforced
Sustainability of Programs or Initiatives
Message Delivery
Funding Given
Q: What are the ways in which you measure your impact now? (Check all that apply)

12. Metrics Ranked Most and Least Valued by
NPOs Mirror Those Attributed to Funders
Q: Please rank the following metrics options in order of importance to your
ORGANIZATION/FUNDERS, with one being the most important, 10 least important.
NPO Both Funders
Numbers Reached,
Attended Events
Actions Taken By
Stakeholders as a Result
of Programs
Progress Achieved
Against Goals
Perceptions Changed or
Reinforced
NPO Both Funders
Funding Received
Funding Given
Message Delivery
Media Coverage
Sustainability of Programs
MOST IMPORTANT*
*Ranked 1 or 2 as “most important” by the majority of survey respondents
*Ranked 9 or 10 as “least important” by the majority of survey respondents
LEAST IMPORTANT*

13. Organizations Revealed Best Practices
for Measurement
Ongoing
review of
progress
against goals
Broad
program
evaluation &
follow-up
Digital
reporting
Tracking
research
advances

14. Secondary Research
Findings
How NPOs Publicly Report Progress Against
Goals via Websites and Annual Reports

15. Cancer NPOs Have a Strong
Commitment to Reporting Results…
91%
Publish an annual
report
89%
Report annual revenue &
operating expenses
60%
Clearly state
organizational goals
46%
Report progress against
each goal

16. …and a Strong Commitment to
Transparency
Approximately 80% identify corporate
and individual sources of grants
Majority report funding given and funding
received (65% and 70%, respectively)
Dollar size, number and impact of grants
both given and received are often disclosed

17. Strong Social Media Presence, Yet Less
Often Measured or Reported
97%
97%
91%91%
60%49%
46%Measure impact on progress based on
social media or web engagement
29%Quantify growth of social media engagement

18. Website Research Shed Light on NPO
Priority Metrics
Audience reach/
event attendance
(91%)
Legislative/
policy work
(74%)
Scientific progress
(63%)
Survey patients about
impact of programs
(23%)
Media coverage (43%)
Message delivery (9%)
Website hits (29%)
Unique visitors (31%)
MORE COMMON LESS COMMON

19. Key Takeaways and Study Implications
Metrics increasingly valued by NPOs, driven largely by funder
expectations
▪ Tasked with finding more immediate measures of success
▪ Limited resources and knowledge on how to best convey impact
Variety of reporting techniques; more “outputs” than “outcomes”
▪ Funders and NPOs largely aligned in metrics they deem most/least important
▪ More focused on reporting results than websites/annual reports indicate
NPO demographics have marginal bearing on metrics activities
▪ Effective metrics reporting not necessarily linked to size or revenues
Diverse stakeholders and funding obstacles will continue to challenge metrics reporting,
but best practices and cost-effective strategies exist to showcase impact

20. Your Guide
to
Intelligent
Giving
Where the
Heart
Meets the
Mind
Critical Friend to Charities

21. The Core Challenge As I See It*
“… there is virtually no
credible evidence
that most nonprofit
organizations
actually produce
any social value.”
*“The End of Charity” by David Hunter – Philadelphia Social Innovations Journal

22. The Nonprofit
Marketplace
Hewlett Fdn
THE ROADMAP TO A SOLUTION TO THE CHALLENGES
INSTITUTIONAL
FUNDERS
supply $ &
technical
assistance for
meaningful
information
BENEFICIARIES
are engaged
&
empowered
to provide
meaningful
information
Institutional
Funders use
a set of
standardized
NP reports
of meaningful
information
Beneficiaries
receive
feedback on
how their
meaningful
information
was utilized

23. WHAT IS MEANINGFUL INFORMATION FOR MEASURING
A CHARITY’S SUCCESS?
RESULTS
GOVERNANCE
FINANCIAL
HEALTH

24. How Does This Fit into the Evolution of our
Rating System?
• CN 1.0 – Financial Health
Launched in 2002 with 1,100 charities
• CN 2.0 – Governance
Launched Sept 20, 2011 with 5,500
charities
• CN 3.0 – Results Reporting
Methodology released and data collection
begun Jan 2013, with a goal of 10,000
charities rated by end of 2016

25. Results Reporting Is…
• An assessment of how charities use their results
internally and share them with stakeholders, including
donors
• We are looking to see that you are reporting on results
measures, and showing how your organization learns
and improves based on those measures (i.e. learning and
improving over time is more important than a ‘snapshot’
of results)

26. How Charity Navigator Evaluates Results Reporting
FIVE ELEMENTS:
1. Alignment of Mission, Solicitations
and Resources
2. Results Logic and Measures
3. Validators
4. Constituent Voice
5. Published Evaluation Reports

27. One Element Will Impact Ratings Next Year
Element One:
Alignment of Mission, Solicitations and Resources

28. Element Two:
Results Logic and Measures
• Is the organization’s causal logic, theory of change,
plausible?
• If applicable, is there an indication of how much of the
action is required to produce the pre-defined outputs and
outcomes?
• Is this logic based on reasonable evidence?
• Are there specified measures (indicators) to be collected
and a plan to do so?

29. Element Three: Validators
• Have your charity’s results been vetted by another
organization?
• Not every charity and cause area will have a validator.
That will not diminish your rating.

30. Element Four: Constituent Voice
• How well does your charity collect and publish
feedback from your primary constituents (the
people who are meant to be the direct recipients of
benefits created by the organization’s actions).
• May not apply to every cause area, but will apply
to most.

31. Element Five:
Published Evaluation Reports
• Does your charity publish evaluation reports that cover
the results of its programs at least every five years?
• Are those reports based on recognized techniques to
understand your results?
• Does you charity explain what, if anything, it is changing
as a result of the findings in the evaluation report?

32. WHAT WE HAVE LEARNED SO FAR
Out of Approximately 1,000 Human Service Charities Evaluated So
Far:
• ~ 70% of Charities have a clearly defined program
• ~ 15% of Charities have a clear definition of how much of
an action is required to produce the pre-defined outputs and
outcomes
• ~2% of Charities have at least 1 of the other 3 elements of Results
Reporting (Validators, Constituent Voice or Published Evaluations)

33. A DREAM THAT YOU CAN PLAY A ROLE IN MAKING A REALITY
RESULTS STATEMENTS,
TODAY TOMORROW
?????

34. To Learn More About How We Measure Success:

35. To Learn More In General About Results Measurement:
Books /Articles/ Studies
1. The Battle for the Soul of the Nonprofit Sector, Berger, Penna and Goldberg, Philadelphia
Social Innovation Journal
2. Money Well Spent by Paul Brest, et. Al
3. Working Hard and Working Well, by David E. K. Hunter
4. Billions of Drops in Millions of Buckets by Goldberg
5. Leap of Reason by Mario Marino
6. The Nonprofit Outcome Toolbox by Dr. Robert Penna
7. Charity Navigator’s webinar on how to use our site
8. Saving Philanthropy Video
9. Independent Sector’s Charting Impact
10. PerformWell Web Site

36. Email: kberger@charitynavigator.org
Blogs:
www.kenscommentary.org
blog.charitynavigator.org
Twitter: kenscommentary
LinkedIn: http://lnkd.in/jKeK5U
Register for Newsletter, Etc.:
https://www.charitynavigator.org/index.cfm?bay=my.login
KEEPING IN TOUCH

37. Evaluation:
Corporate
Perspective
July 2014
Carmen Perez
cperez@cecp.co
212.825.1581

38. THE CEO FORCE
FOR GOOD

39. 1.23%
1.06%
0.98%
0.76%
1.58%
0.96%
0.76%
1.12%
1.25%
1.10%
Utilities (n=18)
Technology (n=25)
Materials (n=16)
Industrials (n=25)
Health Care (n=24)
Financials (n=48)
Energy (n=11)
Consumer Staples (n=20)
Consumer Discretionary (n=26)
Communications (n=10)
2013 Industry Giving Comparisons – Median Giving
as a % of Pre-Tax Profit
For Pharmas:
16.1%

40. Pharmaceuticals Giving at a Glance, 2013
79%
of all companies run a
corporate foundation
86%
of all companies match
employee gifts
62%
of all companies give
internationally
64%
of all companies make
non-cash gifts
78% of
Pharma
companies
89% of
Pharma
companies
100% of
Pharma
companies
78% of
Pharma
companies
N=261 for full sample. Pharma n=9.

41. A Majority of Companies are Measuring the Societal
Value of their Contributions
18%
35%
47%
Very Experienced
(5 Years or More)
Moderately
Experienced
(3-4 Years)
Slightly Experienced
(2 Years or Less)
76% Measure
Outcomes
and/or
Impacts
N=160. N=119.
24% Do Not
Measure
Outcomes
and/or
Impacts

42. Nearly a Third of Companies Focused On Outcome/Impact
Measurement in a Strategic Philanthropic Program
4
2
• For companies only evaluating grants larger than a specific threshold, the average
threshold was approximately $105,000
• “Other” descriptions include companies that:
• Only evaluate foundation grants
• Evaluate for a combination of strategic programs, grant size, and cause area(s)
• Evaluate for as many grants as possible without any specific direction
Description of Impact Measurement Scope
% of
Companies
All grants, regardless of grant size 23%
Only grants made for a strategic philanthropic program(s) 31%
Only grants larger than a specific threshold 16%
Only grants larger than a specific threshold AND made to a specific cause area(s) 12%
Only grants made to a specific cause area (or cause areas) 8%
Other 10%

43. How much do companies report spending on evaluation?
4
3
48%of companies spent
money in 2013 to
evaluate grants
5%
of 2013 contributions were
dedicated to measuring the
outcomes and/or impacts
associated of their grants
Note: Spending money includes contributions to grantees
(earmarked for evaluation) and payments to consultants,
contractors, and evaluation specialists. N=77.
Note: Median. Includes companies spending money
on evaluation. N=37.

44. Companies Use a Variety of Resources for
Measurement Activities in Strategic Giving Programs
26% 26%
20%
15%
11%
9%
7%
Internal
Resource
(Entirely In-
House)
Consulting
Firm(s)
Internal
Resource
(Informed by
External
Resources)
Universities Publicly
Available
Metrics
Research
Institution(s)
Other
N=105.

45. 66% of Companies Partner with Grantees to Measure the
Outcomes and/or Impacts of their Grants
4
5
% of Strategic Programs Focused on:
Common impact metrics for
Education included:
• Literacy Rates
• Graduation Rates
• College Readiness Scores
• Bachelor’s Degree Attainment
Rates
• Racial Disparity Changes
• Attendance Rates
• Teacher Retention Rates
• # of girls/women supported
• # of people obtaining access to
technology
Common impact metrics for
Health & Social Services
included:
• Emergency Room Visits
• Hospital Recidivism
• Immunizations Administered
• Patient Engagement
• Number of children receiving
immunizations
• Increased farmer yields
• Other donors leveraged for
marginalized community
• Improved health resiliency
Common impact metrics for
Community & Economic
Development included:
• Number of Women Impacted
• Credit Score Changes
• Net Worth Changes
• Number of Loans Administered
• Number of Jobs Created
• Business Growth Metrics
• Business Profit Changes
Education: 41% Health: 26% Econ Dev:16%

46. There’s A Lot Out There To Help!
4
6
PSI – Impact Magazine Urban Institute: PerformWell
Health and Safety Section
Center for Disease Control and
Prevention
Steps Communities
CHANGE Communities
RAND Getting to Outcomes 2004
ISBN 0-8330-35285-2
Nurse-Family Partnership Theory
of Change Logic Model
Foundation Center
Tools and Resources for
Assessing Impact

47. Evaluation:
Corporate
Perspective
July 2014
Carmen Perez
cperez@cecp.co
212.825.1581

48. Questions & Answers

49. Grants Announcement
• In partnership with a major university specializing in social innovation
and metrics
• Call for entries open late summer
• NPOs with a healthcare focus and a demonstrated commitment to
metrics reporting eligible
• Submit your email address via post-webinar survey for more
information
The Onyx Advancing Innovation in
Measuring Advocacy Outcomes Award

50. Thank you!