SlideShare a Scribd company logo
TNB Roundtable:
Insights from a Community Foundation’s
Evaluation Director
Mary-Kim Arnold
Director, Evaluation & Learning
January 2016
Agenda
• About the Rhode Island Foundation
• About me and my role
• Your questions
• Key concepts
• Framework for systems work
• Questions / Contact
About the Rhode Island Foundation
• Founded in 1916 – centennial
• 800 million in assets – 35 million in grants (including
donor advised – “discretionary” is closer to 9 million)
• serve the state of RI – largest funder in RI
• www.rifoundation.org
Key concepts
• Levels of accountability
• Units of analysis
• Ratios – meaningful context
• Performance vs. impact
Mountain of Accountability
• Basic accountability for management
• Accountability for “impact”
• Accountability for learning, development, and
adaptation
Units of analysis: who or what?
• Individuals
• Groups
• Geographical units (neighborhood, town, state)
• Social interactions (relationships, social behaviors –
e.g., getting married or divorced)
Ratios: in relation to what*?
“We served 375 students this year
in our after-school program.”
* in relation to what unstated condition, assumption, goal, or expectation?
Meaningful context -- examples
• Of the 375 students who participated in our after-
school program, 98% enrolled for the following year.
behavior
• Of the 400 eligible students in Central Falls, we served
375 in our after-school program. reach
• Our goal for the year was 200 students, which we
surpassed by enrolling 175 students above our goal.
implementation or demand
Meaningful context + unit of analysis
• student: Of the 375 students who participated in our
after-school program, 98% enrolled for the following
year. behavior
• teacher: Teachers in our after-school program had 98%
retention rate for students between years 1 and 2.
performance
• organization: This past year, XYZ reached its 5-year goal
of 98% retention rate for students. implementation
Performance vs. “impact”
▷ High-performing organizations vs. high impact
organizations
▷ Managing for performance but measuring for impact
▷ How might your role and tasks change if you job was
measured for impact?
High performance vs. high impact*
High-performing organization High-impact organization
• fiscally strong
• good governance
• visible, respected leadership
• highly-qualified staff
• clean audit
• growing donor base
• transforming lives
• ending poverty
• closing the achievement gap
• ending hep-C
* Yes! Certainly, a case can be made that only high-performing organizations
can produce results, but the fact that the case can be made
does not eliminate the need to distinguish between the two. **
Systems Change – Areas of Focus
Context
Components
ConnectionsInfrastructure
Scale
Source: www.ccitoolsforfeds.org/doc/AFrameworkForEvaluatingSystemsInitiative.pdf
Defining focus areas of systems work
•Improving the political environment that surrounds a system so it
produces the policy and funding changes needed to create and
sustain it.
CONTEXT
•Establishing high-performing programs and services within the
system that produce results for system beneficiaries.COMPONENTS
•Creating strong and effective linkages across system components
that further improve results for system beneficiaries.CONNECTIONS
•Developing the support systems need to function effectively and
efficiently.INFRASTRUCTURE
•Ensuring a comprehensive system is available to as many people
as possible to produce broad and inclusive results for system
beneficiaries.
SCALE
Theory of Change menu for systems work
Context Components Connections Infrastructure Scale
political will – public
education campaigns,
awareness-building, media
advocacy, community
mobilization, coalition-
building
programs – quality
programs, services, and
interventions
network – linkages
between and among
programs, organizations,
lifecycle, coordination
capacity – of individual
entities or of the network
to function effectively
access – ability to increase
services and numbers
served while maintaining
system integrity
• increases in public
awareness
• public engagement or
mobilization
• reframing issue
• public will
• political will
• new or more diversified
funding or investments
• policy changes
• new programs or
services developed
• expanded individual
programs or coverage
• improved program
quality
• increased program
efficiency
• outcomes that precede
impact
• coordinated
assessments, intake,
outreach
• referrals across services
• coordinated planning
across components
• shared data or
administration
• MOUs between system
components
• entities that oversee and
coordinate subsystems
• funding streams that are
less categorical
• leveraged funds
• systems-level standards
• professional
development structures
• practitioner training and
technical assistance
• system spread
• system depth
• system sustainability
• shifts in system
ownership
DefinitionsAnticipatedOutcomes
Evaluation design menu for systems work
•Has the intervention changed the political environment?
•Has the intervention prompted the availability of new investments, policies, or practices that
will enable changes in the system?
CONTEXT
•Did program activities produce the intended benefits?
•Were interventions implemented as designed?COMPONENTS
•Did the initiative design and implement connections and linkages as intended?
•Did the connections and linkages produce their intended impacts?CONNECTIONS
•Did the initiative establish infrastructure or supports that are consistent with its objectives?
•Did the infrastructure or supports achieve their objectives for effectiveness, sustainability, and
quality?
INFRASTRUCTURE
•Did the initiative enable system scale up with quality and fidelity?
•Did scale-up result in broad impacts for beneficiaries at a system-wide population level?SCALE
Evaluation methodologies menu for systems work
CONTEXT
Theory of
change
evaluation
Case studies
Public polling
Key informant
interviews or
surveys
Media tracking
and analysis
COMPONENTS
Program
evaluation
methodologies
Program
monitoring
Quality
assessments
Efficiency
analysis
Customer
surveys
CONNECTIONS
Program
evaluation
methodologies
System
mapping
Network
analysis
Customer
surveys
INFRASTRUCTURE
Theory of
change
evaluation
Case studies
Performance
audits
Management
information
systems
Practitioner
data collection
SCALE
Population-
based
demographic
analysis
Program
evaluation
methodologies
System or
program
monitoring
Results-based
accountability
Questions? Contact?
mkarnold@rifoundation.org
(401) 427-4009
http://www.rifoundation.org/TransformingRhodeIsland/
FoundationImpact.aspx

More Related Content

What's hot

Gender responsive organizations
Gender responsive organizationsGender responsive organizations
Gender responsive organizations
ILRI
 
CLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENT
CLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENTCLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENT
CLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENT
Orkestra
 
Navigating the Ethical Dilemmas of Building
Navigating the Ethical Dilemmas of BuildingNavigating the Ethical Dilemmas of Building
Navigating the Ethical Dilemmas of Building
Tim Cadman
 
Jeffrey Ireland Resume
Jeffrey Ireland ResumeJeffrey Ireland Resume
Jeffrey Ireland ResumeJeff Ireland
 
Chicago AMA Non-Profit SIG - Maximizing Volunteer Impact
Chicago AMA Non-Profit SIG - Maximizing Volunteer ImpactChicago AMA Non-Profit SIG - Maximizing Volunteer Impact
Chicago AMA Non-Profit SIG - Maximizing Volunteer ImpactChicago AMA
 
Module 3: Gender responsive organizations
Module 3: Gender responsive organizationsModule 3: Gender responsive organizations
Module 3: Gender responsive organizations
ILRI
 
Nonprofit SIG: Maximizing Volunteer Impact
Nonprofit SIG: Maximizing Volunteer ImpactNonprofit SIG: Maximizing Volunteer Impact
Nonprofit SIG: Maximizing Volunteer Impact
Chicago AMA
 
Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09
Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09
Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09
Christopher Coles
 
Quality Is A Verb: Improving Practice In Youth Programs
Quality Is A Verb:  Improving Practice In Youth ProgramsQuality Is A Verb:  Improving Practice In Youth Programs
Quality Is A Verb: Improving Practice In Youth Programs
nmartin7136
 
K12 higher ed collaborations
K12 higher ed collaborationsK12 higher ed collaborations
K12 higher ed collaborations
Minnesota Campus Comapct
 
Maidan summit 2012 Arish Syed, Magic Bus
Maidan summit 2012  Arish Syed, Magic BusMaidan summit 2012  Arish Syed, Magic Bus
Maidan summit 2012 Arish Syed, Magic BusMaidan.in
 
AMCHP 2015: Care Coordination for CYSHCN
AMCHP 2015: Care Coordination for CYSHCNAMCHP 2015: Care Coordination for CYSHCN
AMCHP 2015: Care Coordination for CYSHCN
LucilePackardFoundation
 
VolunteerNet
VolunteerNetVolunteerNet
VolunteerNetrehamraja
 
AEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations
AEA 2007 Kuo Performance Metrics and Evaluation in Large FoundationsAEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations
AEA 2007 Kuo Performance Metrics and Evaluation in Large FoundationsVictor Kuo
 
Summit Highlights (By The Lee Institute)
Summit Highlights (By The Lee Institute)Summit Highlights (By The Lee Institute)
Summit Highlights (By The Lee Institute)
South Carolina First Steps
 
2018 First 5 California Summit Presentation: Creating a Strategic Approach to...
2018 First 5 California Summit Presentation: Creating a Strategic Approach to...2018 First 5 California Summit Presentation: Creating a Strategic Approach to...
2018 First 5 California Summit Presentation: Creating a Strategic Approach to...
appliedsurveyresearch
 
The "Hub and Spoke" Evaluation 2013-17
The "Hub and Spoke" Evaluation 2013-17The "Hub and Spoke" Evaluation 2013-17
The "Hub and Spoke" Evaluation 2013-17
BASPCAN
 
Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...
Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...
Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...
Independent Science and Partnership Council of the CGIAR
 

What's hot (20)

Gender responsive organizations
Gender responsive organizationsGender responsive organizations
Gender responsive organizations
 
CLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENT
CLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENTCLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENT
CLUSTER EVALUATION KEY TO UNLOCKING LEARNING AND IMPROVEMENT
 
Navigating the Ethical Dilemmas of Building
Navigating the Ethical Dilemmas of BuildingNavigating the Ethical Dilemmas of Building
Navigating the Ethical Dilemmas of Building
 
Jeffrey Ireland Resume
Jeffrey Ireland ResumeJeffrey Ireland Resume
Jeffrey Ireland Resume
 
Chicago AMA Non-Profit SIG - Maximizing Volunteer Impact
Chicago AMA Non-Profit SIG - Maximizing Volunteer ImpactChicago AMA Non-Profit SIG - Maximizing Volunteer Impact
Chicago AMA Non-Profit SIG - Maximizing Volunteer Impact
 
Module 3: Gender responsive organizations
Module 3: Gender responsive organizationsModule 3: Gender responsive organizations
Module 3: Gender responsive organizations
 
Nonprofit SIG: Maximizing Volunteer Impact
Nonprofit SIG: Maximizing Volunteer ImpactNonprofit SIG: Maximizing Volunteer Impact
Nonprofit SIG: Maximizing Volunteer Impact
 
Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09
Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09
Un fao sofa 2010 gender and value chains coles mitchell 15 sept 09
 
Quality Is A Verb: Improving Practice In Youth Programs
Quality Is A Verb:  Improving Practice In Youth ProgramsQuality Is A Verb:  Improving Practice In Youth Programs
Quality Is A Verb: Improving Practice In Youth Programs
 
K12 higher ed collaborations
K12 higher ed collaborationsK12 higher ed collaborations
K12 higher ed collaborations
 
Maidan summit 2012 Arish Syed, Magic Bus
Maidan summit 2012  Arish Syed, Magic BusMaidan summit 2012  Arish Syed, Magic Bus
Maidan summit 2012 Arish Syed, Magic Bus
 
AA Resume 2016
AA Resume 2016AA Resume 2016
AA Resume 2016
 
AMCHP 2015: Care Coordination for CYSHCN
AMCHP 2015: Care Coordination for CYSHCNAMCHP 2015: Care Coordination for CYSHCN
AMCHP 2015: Care Coordination for CYSHCN
 
VolunteerNet
VolunteerNetVolunteerNet
VolunteerNet
 
AEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations
AEA 2007 Kuo Performance Metrics and Evaluation in Large FoundationsAEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations
AEA 2007 Kuo Performance Metrics and Evaluation in Large Foundations
 
Summit Highlights (By The Lee Institute)
Summit Highlights (By The Lee Institute)Summit Highlights (By The Lee Institute)
Summit Highlights (By The Lee Institute)
 
2018 First 5 California Summit Presentation: Creating a Strategic Approach to...
2018 First 5 California Summit Presentation: Creating a Strategic Approach to...2018 First 5 California Summit Presentation: Creating a Strategic Approach to...
2018 First 5 California Summit Presentation: Creating a Strategic Approach to...
 
Gender audit feb 2013
Gender audit feb 2013Gender audit feb 2013
Gender audit feb 2013
 
The "Hub and Spoke" Evaluation 2013-17
The "Hub and Spoke" Evaluation 2013-17The "Hub and Spoke" Evaluation 2013-17
The "Hub and Spoke" Evaluation 2013-17
 
Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...
Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...
Jody Harris, IFPRI and LCIRAH "Assessing Intersectoral Coordination for Nutri...
 

Similar to TNB Roundtable slide deck by Mary-Kim Arnold of Rhode Island Foundation

CSR Impact Measurement & Management
CSR Impact Measurement & ManagementCSR Impact Measurement & Management
CSR Impact Measurement & Management
Wayne Dunn
 
Week3 rainey chapter_6
Week3 rainey chapter_6Week3 rainey chapter_6
Week3 rainey chapter_6
mmzzmartinez
 
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
Clear Impact
 
7 Measures of Success: What Remarkable Associations Do That Others Don't
7 Measures of Success: What Remarkable Associations Do That Others Don't7 Measures of Success: What Remarkable Associations Do That Others Don't
7 Measures of Success: What Remarkable Associations Do That Others Don't
Greg Melia, CAE
 
Tools for Measuring Place-Based Systems Change
Tools for Measuring Place-Based Systems ChangeTools for Measuring Place-Based Systems Change
Tools for Measuring Place-Based Systems Change
Justin Piff
 
ADRP | Measurement of Success
ADRP | Measurement of SuccessADRP | Measurement of Success
ADRP | Measurement of Success
Heurista, Co.
 
Organizational Capacity-Building Series - Session 7: Strategic Planning
 Organizational Capacity-Building Series - Session 7: Strategic Planning Organizational Capacity-Building Series - Session 7: Strategic Planning
Organizational Capacity-Building Series - Session 7: Strategic Planning
INGENAES
 
Systems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINS
Systems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINSSystems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINS
Systems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINS
CORE Group
 
Charity Navigator 2.0 Case Study Presentation
Charity Navigator 2.0 Case Study PresentationCharity Navigator 2.0 Case Study Presentation
Charity Navigator 2.0 Case Study Presentation
CharityNav
 
Introduction to Policy Evaluation
Introduction to Policy EvaluationIntroduction to Policy Evaluation
Introduction to Policy Evaluation
pasicUganda
 
6224608.ppt
6224608.ppt6224608.ppt
6224608.ppt
drjeetasarkar
 
New Frameworks for Measuring Capacity and Assessing Performance
New Frameworks for Measuring Capacity and Assessing PerformanceNew Frameworks for Measuring Capacity and Assessing Performance
New Frameworks for Measuring Capacity and Assessing Performance
TCC Group
 
sociology-for-mineral-exploration-1.pptx
sociology-for-mineral-exploration-1.pptxsociology-for-mineral-exploration-1.pptx
sociology-for-mineral-exploration-1.pptx
LeonardoOctavioOlart
 
Meaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matterMeaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matter
Rebecca Jones
 
PDE Week 3 Developing and evaluating programs using the logic model
PDE Week 3 Developing and evaluating programs using the logic modelPDE Week 3 Developing and evaluating programs using the logic model
PDE Week 3 Developing and evaluating programs using the logic model
kpravera
 
Measures of Effective Comprehensive Community Action
Measures of Effective Comprehensive Community ActionMeasures of Effective Comprehensive Community Action
Measures of Effective Comprehensive Community ActionDavid Knight
 
Measuring the Impact of Your Nonprofit
Measuring the Impact of Your NonprofitMeasuring the Impact of Your Nonprofit
Measuring the Impact of Your Nonprofit
NetSquared Vancouver
 
18.07.12 LLC Webinar Series l Leadership and Large Scale Change
18.07.12 LLC Webinar Series l Leadership and Large Scale Change18.07.12 LLC Webinar Series l Leadership and Large Scale Change
18.07.12 LLC Webinar Series l Leadership and Large Scale Change
Leadership Learning Community
 
Organizational_Development_pptx.pptx
Organizational_Development_pptx.pptxOrganizational_Development_pptx.pptx
Organizational_Development_pptx.pptx
Novalon
 
Organizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program EvaluationOrganizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program Evaluation
INGENAES
 

Similar to TNB Roundtable slide deck by Mary-Kim Arnold of Rhode Island Foundation (20)

CSR Impact Measurement & Management
CSR Impact Measurement & ManagementCSR Impact Measurement & Management
CSR Impact Measurement & Management
 
Week3 rainey chapter_6
Week3 rainey chapter_6Week3 rainey chapter_6
Week3 rainey chapter_6
 
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
 
7 Measures of Success: What Remarkable Associations Do That Others Don't
7 Measures of Success: What Remarkable Associations Do That Others Don't7 Measures of Success: What Remarkable Associations Do That Others Don't
7 Measures of Success: What Remarkable Associations Do That Others Don't
 
Tools for Measuring Place-Based Systems Change
Tools for Measuring Place-Based Systems ChangeTools for Measuring Place-Based Systems Change
Tools for Measuring Place-Based Systems Change
 
ADRP | Measurement of Success
ADRP | Measurement of SuccessADRP | Measurement of Success
ADRP | Measurement of Success
 
Organizational Capacity-Building Series - Session 7: Strategic Planning
 Organizational Capacity-Building Series - Session 7: Strategic Planning Organizational Capacity-Building Series - Session 7: Strategic Planning
Organizational Capacity-Building Series - Session 7: Strategic Planning
 
Systems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINS
Systems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINSSystems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINS
Systems Thinking: Working Backwards, Not Backwards Thinking HENDRIX-JENKINS
 
Charity Navigator 2.0 Case Study Presentation
Charity Navigator 2.0 Case Study PresentationCharity Navigator 2.0 Case Study Presentation
Charity Navigator 2.0 Case Study Presentation
 
Introduction to Policy Evaluation
Introduction to Policy EvaluationIntroduction to Policy Evaluation
Introduction to Policy Evaluation
 
6224608.ppt
6224608.ppt6224608.ppt
6224608.ppt
 
New Frameworks for Measuring Capacity and Assessing Performance
New Frameworks for Measuring Capacity and Assessing PerformanceNew Frameworks for Measuring Capacity and Assessing Performance
New Frameworks for Measuring Capacity and Assessing Performance
 
sociology-for-mineral-exploration-1.pptx
sociology-for-mineral-exploration-1.pptxsociology-for-mineral-exploration-1.pptx
sociology-for-mineral-exploration-1.pptx
 
Meaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matterMeaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matter
 
PDE Week 3 Developing and evaluating programs using the logic model
PDE Week 3 Developing and evaluating programs using the logic modelPDE Week 3 Developing and evaluating programs using the logic model
PDE Week 3 Developing and evaluating programs using the logic model
 
Measures of Effective Comprehensive Community Action
Measures of Effective Comprehensive Community ActionMeasures of Effective Comprehensive Community Action
Measures of Effective Comprehensive Community Action
 
Measuring the Impact of Your Nonprofit
Measuring the Impact of Your NonprofitMeasuring the Impact of Your Nonprofit
Measuring the Impact of Your Nonprofit
 
18.07.12 LLC Webinar Series l Leadership and Large Scale Change
18.07.12 LLC Webinar Series l Leadership and Large Scale Change18.07.12 LLC Webinar Series l Leadership and Large Scale Change
18.07.12 LLC Webinar Series l Leadership and Large Scale Change
 
Organizational_Development_pptx.pptx
Organizational_Development_pptx.pptxOrganizational_Development_pptx.pptx
Organizational_Development_pptx.pptx
 
Organizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program EvaluationOrganizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program Evaluation
 

Recently uploaded

2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended
2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended
2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended
johnmarimigallon
 
A proposed request for information on LIHTC
A proposed request for information on LIHTCA proposed request for information on LIHTC
A proposed request for information on LIHTC
Roger Valdez
 
Invitation Letter for an alumni association
Invitation Letter for an alumni associationInvitation Letter for an alumni association
Invitation Letter for an alumni association
elmerdalida001
 
如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样
如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样
如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样
850fcj96
 
A guide to the International day of Potatoes 2024 - May 30th
A guide to the International day of Potatoes 2024 - May 30thA guide to the International day of Potatoes 2024 - May 30th
A guide to the International day of Potatoes 2024 - May 30th
Christina Parmionova
 
Transit-Oriented Development Study Working Group Meeting
Transit-Oriented Development Study Working Group MeetingTransit-Oriented Development Study Working Group Meeting
Transit-Oriented Development Study Working Group Meeting
Cuyahoga County Planning Commission
 
NHAI_Under_Implementation_01-05-2024.pdf
NHAI_Under_Implementation_01-05-2024.pdfNHAI_Under_Implementation_01-05-2024.pdf
NHAI_Under_Implementation_01-05-2024.pdf
AjayVejendla3
 
RFP for Reno's Community Assistance Center
RFP for Reno's Community Assistance CenterRFP for Reno's Community Assistance Center
RFP for Reno's Community Assistance Center
This Is Reno
 
CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054
CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054
CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054
Congressional Budget Office
 
About Potato, The scientific name of the plant is Solanum tuberosum (L).
About Potato, The scientific name of the plant is Solanum tuberosum (L).About Potato, The scientific name of the plant is Solanum tuberosum (L).
About Potato, The scientific name of the plant is Solanum tuberosum (L).
Christina Parmionova
 
Get Government Grants and Assistance Program
Get Government Grants and Assistance ProgramGet Government Grants and Assistance Program
Get Government Grants and Assistance Program
Get Government Grants
 
快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样
快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样
快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样
850fcj96
 
Monitoring Health for the SDGs - Global Health Statistics 2024 - WHO
Monitoring Health for the SDGs - Global Health Statistics 2024 - WHOMonitoring Health for the SDGs - Global Health Statistics 2024 - WHO
Monitoring Health for the SDGs - Global Health Statistics 2024 - WHO
Christina Parmionova
 
A Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC Charlotte
A Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC CharlotteA Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC Charlotte
A Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC Charlotte
University of North Carolina at Charlotte
 
Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...
Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...
Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...
OECDregions
 
2024: The FAR - Federal Acquisition Regulations, Part 38
2024: The FAR - Federal Acquisition Regulations, Part 382024: The FAR - Federal Acquisition Regulations, Part 38
2024: The FAR - Federal Acquisition Regulations, Part 38
JSchaus & Associates
 
Donate to charity during this holiday season
Donate to charity during this holiday seasonDonate to charity during this holiday season
Donate to charity during this holiday season
SERUDS INDIA
 
2024: The FAR - Federal Acquisition Regulations, Part 39
2024: The FAR - Federal Acquisition Regulations, Part 392024: The FAR - Federal Acquisition Regulations, Part 39
2024: The FAR - Federal Acquisition Regulations, Part 39
JSchaus & Associates
 
State crafting: Changes and challenges for managing the public finances
State crafting: Changes and challenges for managing the public financesState crafting: Changes and challenges for managing the public finances
State crafting: Changes and challenges for managing the public finances
ResolutionFoundation
 
Opinions on EVs: Metro Atlanta Speaks 2023
Opinions on EVs: Metro Atlanta Speaks 2023Opinions on EVs: Metro Atlanta Speaks 2023
Opinions on EVs: Metro Atlanta Speaks 2023
ARCResearch
 

Recently uploaded (20)

2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended
2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended
2017 Omnibus Rules on Appointments and Other Human Resource Actions, As Amended
 
A proposed request for information on LIHTC
A proposed request for information on LIHTCA proposed request for information on LIHTC
A proposed request for information on LIHTC
 
Invitation Letter for an alumni association
Invitation Letter for an alumni associationInvitation Letter for an alumni association
Invitation Letter for an alumni association
 
如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样
如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样
如何办理(uoit毕业证书)加拿大安大略理工大学毕业证文凭证书录取通知原版一模一样
 
A guide to the International day of Potatoes 2024 - May 30th
A guide to the International day of Potatoes 2024 - May 30thA guide to the International day of Potatoes 2024 - May 30th
A guide to the International day of Potatoes 2024 - May 30th
 
Transit-Oriented Development Study Working Group Meeting
Transit-Oriented Development Study Working Group MeetingTransit-Oriented Development Study Working Group Meeting
Transit-Oriented Development Study Working Group Meeting
 
NHAI_Under_Implementation_01-05-2024.pdf
NHAI_Under_Implementation_01-05-2024.pdfNHAI_Under_Implementation_01-05-2024.pdf
NHAI_Under_Implementation_01-05-2024.pdf
 
RFP for Reno's Community Assistance Center
RFP for Reno's Community Assistance CenterRFP for Reno's Community Assistance Center
RFP for Reno's Community Assistance Center
 
CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054
CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054
CBO’s Outlook for U.S. Fertility Rates: 2024 to 2054
 
About Potato, The scientific name of the plant is Solanum tuberosum (L).
About Potato, The scientific name of the plant is Solanum tuberosum (L).About Potato, The scientific name of the plant is Solanum tuberosum (L).
About Potato, The scientific name of the plant is Solanum tuberosum (L).
 
Get Government Grants and Assistance Program
Get Government Grants and Assistance ProgramGet Government Grants and Assistance Program
Get Government Grants and Assistance Program
 
快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样
快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样
快速制作(ocad毕业证书)加拿大安大略艺术设计学院毕业证本科学历雅思成绩单原版一模一样
 
Monitoring Health for the SDGs - Global Health Statistics 2024 - WHO
Monitoring Health for the SDGs - Global Health Statistics 2024 - WHOMonitoring Health for the SDGs - Global Health Statistics 2024 - WHO
Monitoring Health for the SDGs - Global Health Statistics 2024 - WHO
 
A Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC Charlotte
A Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC CharlotteA Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC Charlotte
A Guide to AI for Smarter Nonprofits - Dr. Cori Faklaris, UNC Charlotte
 
Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...
Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...
Preliminary findings _OECD field visits to ten regions in the TSI EU mining r...
 
2024: The FAR - Federal Acquisition Regulations, Part 38
2024: The FAR - Federal Acquisition Regulations, Part 382024: The FAR - Federal Acquisition Regulations, Part 38
2024: The FAR - Federal Acquisition Regulations, Part 38
 
Donate to charity during this holiday season
Donate to charity during this holiday seasonDonate to charity during this holiday season
Donate to charity during this holiday season
 
2024: The FAR - Federal Acquisition Regulations, Part 39
2024: The FAR - Federal Acquisition Regulations, Part 392024: The FAR - Federal Acquisition Regulations, Part 39
2024: The FAR - Federal Acquisition Regulations, Part 39
 
State crafting: Changes and challenges for managing the public finances
State crafting: Changes and challenges for managing the public financesState crafting: Changes and challenges for managing the public finances
State crafting: Changes and challenges for managing the public finances
 
Opinions on EVs: Metro Atlanta Speaks 2023
Opinions on EVs: Metro Atlanta Speaks 2023Opinions on EVs: Metro Atlanta Speaks 2023
Opinions on EVs: Metro Atlanta Speaks 2023
 

TNB Roundtable slide deck by Mary-Kim Arnold of Rhode Island Foundation

  • 1. TNB Roundtable: Insights from a Community Foundation’s Evaluation Director Mary-Kim Arnold Director, Evaluation & Learning January 2016
  • 2. Agenda • About the Rhode Island Foundation • About me and my role • Your questions • Key concepts • Framework for systems work • Questions / Contact
  • 3. About the Rhode Island Foundation • Founded in 1916 – centennial • 800 million in assets – 35 million in grants (including donor advised – “discretionary” is closer to 9 million) • serve the state of RI – largest funder in RI • www.rifoundation.org
  • 4. Key concepts • Levels of accountability • Units of analysis • Ratios – meaningful context • Performance vs. impact
  • 5.
  • 6. Mountain of Accountability • Basic accountability for management • Accountability for “impact” • Accountability for learning, development, and adaptation
  • 7. Units of analysis: who or what? • Individuals • Groups • Geographical units (neighborhood, town, state) • Social interactions (relationships, social behaviors – e.g., getting married or divorced)
  • 8. Ratios: in relation to what*? “We served 375 students this year in our after-school program.” * in relation to what unstated condition, assumption, goal, or expectation?
  • 9. Meaningful context -- examples • Of the 375 students who participated in our after- school program, 98% enrolled for the following year. behavior • Of the 400 eligible students in Central Falls, we served 375 in our after-school program. reach • Our goal for the year was 200 students, which we surpassed by enrolling 175 students above our goal. implementation or demand
  • 10. Meaningful context + unit of analysis • student: Of the 375 students who participated in our after-school program, 98% enrolled for the following year. behavior • teacher: Teachers in our after-school program had 98% retention rate for students between years 1 and 2. performance • organization: This past year, XYZ reached its 5-year goal of 98% retention rate for students. implementation
  • 11. Performance vs. “impact” ▷ High-performing organizations vs. high impact organizations ▷ Managing for performance but measuring for impact ▷ How might your role and tasks change if you job was measured for impact?
  • 12. High performance vs. high impact* High-performing organization High-impact organization • fiscally strong • good governance • visible, respected leadership • highly-qualified staff • clean audit • growing donor base • transforming lives • ending poverty • closing the achievement gap • ending hep-C * Yes! Certainly, a case can be made that only high-performing organizations can produce results, but the fact that the case can be made does not eliminate the need to distinguish between the two. **
  • 13. Systems Change – Areas of Focus Context Components ConnectionsInfrastructure Scale Source: www.ccitoolsforfeds.org/doc/AFrameworkForEvaluatingSystemsInitiative.pdf
  • 14. Defining focus areas of systems work •Improving the political environment that surrounds a system so it produces the policy and funding changes needed to create and sustain it. CONTEXT •Establishing high-performing programs and services within the system that produce results for system beneficiaries.COMPONENTS •Creating strong and effective linkages across system components that further improve results for system beneficiaries.CONNECTIONS •Developing the support systems need to function effectively and efficiently.INFRASTRUCTURE •Ensuring a comprehensive system is available to as many people as possible to produce broad and inclusive results for system beneficiaries. SCALE
  • 15. Theory of Change menu for systems work Context Components Connections Infrastructure Scale political will – public education campaigns, awareness-building, media advocacy, community mobilization, coalition- building programs – quality programs, services, and interventions network – linkages between and among programs, organizations, lifecycle, coordination capacity – of individual entities or of the network to function effectively access – ability to increase services and numbers served while maintaining system integrity • increases in public awareness • public engagement or mobilization • reframing issue • public will • political will • new or more diversified funding or investments • policy changes • new programs or services developed • expanded individual programs or coverage • improved program quality • increased program efficiency • outcomes that precede impact • coordinated assessments, intake, outreach • referrals across services • coordinated planning across components • shared data or administration • MOUs between system components • entities that oversee and coordinate subsystems • funding streams that are less categorical • leveraged funds • systems-level standards • professional development structures • practitioner training and technical assistance • system spread • system depth • system sustainability • shifts in system ownership DefinitionsAnticipatedOutcomes
  • 16. Evaluation design menu for systems work •Has the intervention changed the political environment? •Has the intervention prompted the availability of new investments, policies, or practices that will enable changes in the system? CONTEXT •Did program activities produce the intended benefits? •Were interventions implemented as designed?COMPONENTS •Did the initiative design and implement connections and linkages as intended? •Did the connections and linkages produce their intended impacts?CONNECTIONS •Did the initiative establish infrastructure or supports that are consistent with its objectives? •Did the infrastructure or supports achieve their objectives for effectiveness, sustainability, and quality? INFRASTRUCTURE •Did the initiative enable system scale up with quality and fidelity? •Did scale-up result in broad impacts for beneficiaries at a system-wide population level?SCALE
  • 17. Evaluation methodologies menu for systems work CONTEXT Theory of change evaluation Case studies Public polling Key informant interviews or surveys Media tracking and analysis COMPONENTS Program evaluation methodologies Program monitoring Quality assessments Efficiency analysis Customer surveys CONNECTIONS Program evaluation methodologies System mapping Network analysis Customer surveys INFRASTRUCTURE Theory of change evaluation Case studies Performance audits Management information systems Practitioner data collection SCALE Population- based demographic analysis Program evaluation methodologies System or program monitoring Results-based accountability

Editor's Notes

  1. Blandin Foundation – Mountain of Accountability
  2. The important part of this is not so much that we have to focus on one or another one of these elements in our portfolios, but rather these are what help us get to how we should evaluate – what we should be looking for. So, in initiatives where we are looking at context – e.g., Backyard – we are looking for changes in the political environment – shifts in investment, policy or practice that will enable changes at other levels – so theory of change is that
  3. These are possible methodologies – not necessary to use all or even any listed – a menu of options for methodologies that align for each focus area.