MEASURING PROGRESS, WITH APURPOSEMaytree’s Five Good Ideas SeriesApril 18, 2012Blair DimockDirector, Research, Evaluation ...
MEASURING PROGRESS, WITH A PURPOSE1. Map to the Why: measuring progress is mission critical.2. From accountability to acti...
1. MAP TO THE WHY: MEASURING PROGRESS IS            MISSION CRITICAL
FUTURE FUND: THEORY OF CHANGEIf we make significant, long-term investments in aportfolio of innovative initiatives, and su...
2. FROM   ACCOUNTABILITY TO ACTION LEARNING:            THINKING “IF…, THEN…”.
Framing Question #1How can we learn DURING our grantmaking work?                                            “When someone ...
Emergent Learning Map:   Framing Question: How can we…?                              What will it take to…?    What we’ve ...
OTF Example:             What will it take to build the capacity of the environment sector, in                            ...
Action Review Cycle (ARC)
ACTION REVIEW CYCLE                                                                     Facilitator:    Action    Review  ...
Framing QuestionHow can we tackle the sheer volume of whatthere is to learn?“If you try to learn everything, you’re going ...
Creating a Learning Agenda               Framing Question  Hypothesis        Hypothesis        Hypothesis               Ac...
BUILDING THE CAPACITY OF THE ENVIRONMENT SECTORIf we…                                      Then we will…connect advocates ...
3. ASK   THE RIGHT QUESTIONS.
CHARTING IMPACT: THE 5 QUESTIONS1. What is your organization aiming to accomplish?2. What are your strategies for making t...
4. MAKE   WHAT YOU MEASURE WORK FOR YOU.
THE   EVALUATION HIERARCHY
Numerous Sources+ Multiple Methods+ Different Points of View= High Degree of Confidence
5. IN OUR WORLD OF NETWORKS, ENGAGE.
FROM ACTIVITIES TO DELIBERATE LEARNING“This all takes a level of discipline that’s hard to maintain. We can comeup with th...
RESOURCES1. Jim Collins, “Good to Great for the Sectors”, http://www.jimcollins.com/books/g2g-   ss.html; Grantcraft, “Map...
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012
Upcoming SlideShare
Loading in …5
×

Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012

1,933 views

Published on

Measuring our progress in achieving our goals and fulfilling our missions is more important than ever. In a world of economic volatility, government constraint and increasing transparency, funders and their grantees need more effective ways to demonstrate their individual and collective impact to a broadening array of interested stakeholders.

Blair Dimock shared the steps they have taken at the Ontario Trillium Foundation has taken to re-invent how they measure the impact of their granting, what they measure, and why. Through a focus on balancing accountability with an action learning agenda, using mixed measurement methods, increasing engagement with grantees, staff and volunteers, and experimentation, you, too can improve how you map your progress towards achieving your organization’s mission.

Find out more about Five Good Ideas: http://maytree.com/training/fivegoodideas

Watch a video of Blair's presentation - http://maytree.com/fgi/mapping-progress-with-a-purpose.html

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,933
On SlideShare
0
From Embeds
0
Number of Embeds
651
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Govt agency$110M, 1500 grants per year – small, time-limited grants, 35% capacity-building, 25+% small capital grants (many under $15K)Decentralized, place-based, volunteer decision-making significant implications for evaluation and learning – accy. vs. grantmaking effectiveness  shift to emphasis on continuous learning tied to enhanced impact – but we have to remain accountable (public dollars)
  • Notes from Learning in the Thick of It A beforeaction review (BAR), requires teams to answer four questions before embarking on an important action: What are our intended results and measures?What challenges can we anticipate?What have we or others learned from similar situations? What will make us successful this time?  The responses to those questions align the team’s objectives and set the stage for an effective AAR meeting following the action. In addition, breaking projects into smaller chunks, bookended by short BAR and AAR meetings conducted in task-focused groups, establishes feedback loops that can help a project team maximize performance and develop a learning culture over time. Every organization, every team, and every project will likely require different levels of preparation, execution, and review. However, we have distilled some best practices from the few companies we studied that use AARs well. For example, leaders should phase in an AAR regimen, beginning with the most important and complex work their business units perform. Teams should commit to holding short BAR and AAR meetings as they go, keeping things simple at first and developing the process slowly—adding rehearsals, knowledgesharing activities and systems, richer metrics, and other features dictated by the particular practice. While companies will differ on the specifics they adopt, four fundamentals of the OPFOR process are mandatory. Lessons must first and foremost benefit the team that extracts them. The AAR process must start at the beginning of the activity. Lessons must link explicitly to future actions. And leaders must hold everyone, especially themselves, accountable for learning. In a fastchanging environment, the capacity to learn lessons is more valuable than any individual lesson learned.
  • Frequently, the initial hypotheses underlying a strategy that is intended to bring about change in a complex system are imperfect. The faster a grant maker can learn what is and is not working and improve its strategy, the greater its impact is likely to be. Evaluation can be one vital source of information to support this learning.
  • FF Eval 2011-12Applicant survey – 223 respondents, 165 completed, 30 granteesStaff and Volunteer Survey – 28 respondents19 Key Informant Interviews 17 Emergent Learning Sessions2 staff focus groups Internal grant analysisLiterature review
  • There is no single data set for measuring Foundation-wide effectiveness.Look for indicators that taken together can suggest how well you are doing.
  • Five Good Ideas with Blair Dimock: Mapping Progress, with a Purpose - April 18, 2012

    1. 1. MEASURING PROGRESS, WITH APURPOSEMaytree’s Five Good Ideas SeriesApril 18, 2012Blair DimockDirector, Research, Evaluation and KnowledgeManagementOntario Trillium Foundationbdimock@otf.ca
    2. 2. MEASURING PROGRESS, WITH A PURPOSE1. Map to the Why: measuring progress is mission critical.2. From accountability to action learning: thinking “if…,then…”.3. Ask the right questions.4. Make what you measure work for you.5. In our world of networks, engage.
    3. 3. 1. MAP TO THE WHY: MEASURING PROGRESS IS MISSION CRITICAL
    4. 4. FUTURE FUND: THEORY OF CHANGEIf we make significant, long-term investments in aportfolio of innovative initiatives, and support themthrough high engagement relationships and networking,we will be a catalyst for transformational change.
    5. 5. 2. FROM ACCOUNTABILITY TO ACTION LEARNING: THINKING “IF…, THEN…”.
    6. 6. Framing Question #1How can we learn DURING our grantmaking work? “When someone Medium Cycle says, ‘we should learn,’ everyone nods. The problem is that it’s not specific. The intention to learn, by itself, is not that helpful.”
    7. 7. Emergent Learning Map: Framing Question: How can we…? What will it take to…? What we’ve What we think learned from what will make us has already successful in the happened future Insights Hypotheses Ground Truth Opportunities Key moments Upcoming looking back from opportunities to test which we can our hypotheses in learn action
    8. 8. OTF Example: What will it take to build the capacity of the environment sector, in order to increase its impact? OTF can play a valued role by strengthening the links between Capacity building If we introduce a If we invest in innovative grass-roots and large requires a long- “high engagement” collaborations, the capacity of environmental non-government term commitment. approach to the sector will be enhanced, organizations. and its impact will increase. evaluation and monitoring, we will learn more Granting in high effectively and Granting widely and for short- volume leaves little If we take a portfolio approach to increase our term projects may lessen our time or resources for grantmaking decisions, the impact likelihood of impact and not lead to lasting effective learning. of the initiative will be greater. success. results. Insights Hypotheses The environment sector in Ground Truth Opportunities Ontario is smaller, less developed and has lower Future Fund Round 1 The sector is made up of capacity than the other a few large, high sectors we fund. capacity organizations Design of evaluation High engagement and many small staff team organizations who lack plan As the largest funder of capacity in key areas. environmental organizations in Ontario, Learning circles (grantees OTF has an opportunity to and staff team) be a leader in helping the sector achieve greater Future Fund impact. Round 2
    9. 9. Action Review Cycle (ARC)
    10. 10. ACTION REVIEW CYCLE Facilitator: Action Review CycleTask:Team:Date: Before Action ReviewWhat is our intent (purpose and desired result)?How will we measure success?What challenges can we anticipate?What did we/others learn in similar situations?What will make the biggest difference this time? ActionDate: After Action ReviewWhat were our results? (Intended vs. Actual)What caused these results?What will we sustain? What will we improve?Next opportunities: (When are the next opportunities to use and refine what we learned?)Notes: (Who we should copy this to; other action items; new burning questions; etc.)
    11. 11. Framing QuestionHow can we tackle the sheer volume of whatthere is to learn?“If you try to learn everything, you’re going to drown. We want to learnfrom every grant; every program; every event. It’s too much. The biggestchallenge is to figure out what the most strategic things to be learned areand letting go of the rest of it.” -- Mary Williams, Lumina Foundation for Education
    12. 12. Creating a Learning Agenda Framing Question Hypothesis Hypothesis Hypothesis Action Learning Plan
    13. 13. BUILDING THE CAPACITY OF THE ENVIRONMENT SECTORIf we… Then we will…connect advocates about early influence practice and policies forenvironmental exposure to toxins with chronic disease prevention.public health practitionersengage a broad range of environmental Strengthen the policy effectiveness oforganizations in the setting of shared the sector.environmental priorities for Ontariolink regional efforts in the northern and build the capacity of communities tosouthern parts of Ontario through a respond to common climate-changeNorth-South Climate Change Network challenges.build a provincial alliance to address support a new generation of viable,issues of farmland access and ecological, local farmers.successionfoster partnerships between Community assure the protection of up to 50% ofFoundations and Land Trusts to raise currently owned conservation lands.stewardship fundsinspire environmental non-profits to transform the sector in terms of itsembrace diversity in their audiences ethno-cultural and racial diversity.and within their organizations
    14. 14. 3. ASK THE RIGHT QUESTIONS.
    15. 15. CHARTING IMPACT: THE 5 QUESTIONS1. What is your organization aiming to accomplish?2. What are your strategies for making this happen?3. What are your organization’s capabilities for doing this?4. How will your organization know if you are making progress?5. What have and haven’t you accomplished so far?
    16. 16. 4. MAKE WHAT YOU MEASURE WORK FOR YOU.
    17. 17. THE EVALUATION HIERARCHY
    18. 18. Numerous Sources+ Multiple Methods+ Different Points of View= High Degree of Confidence
    19. 19. 5. IN OUR WORLD OF NETWORKS, ENGAGE.
    20. 20. FROM ACTIVITIES TO DELIBERATE LEARNING“This all takes a level of discipline that’s hard to maintain. We can comeup with the greatest dashboards, logic models, learning agendas. It takesdiscipline to look at them and say, ‘What is our outcome?’” -- Jane Donahue, Deaconess Foundation
    21. 21. RESOURCES1. Jim Collins, “Good to Great for the Sectors”, http://www.jimcollins.com/books/g2g- ss.html; Grantcraft, “Mapping Change”, http://www.grantcraft.org/index.cfm?fuseaction=Page.ViewPage&pageId=1542.2. 4th Quadrant Partners, “Where Learning turns into Results”, http://www.4qpartners.com/Tools.html; International Development Research Centre, “Tools and Training”, http://www.idrc.ca/EN/Resources/Tools_and_Training/Pages/default.aspx.3. Charting Impact, “The 5 Questions”, http://www.chartingimpact.org/complete- your-report/five-questions/; Social Asset Measurements “Non-Profit and Charitable Solutions”, http://www.socialassets.org/.4. Grantcraft, “Making Measures work for You”, http://www.grantcraft.org/index.cfm?fuseaction=Page.ViewPage&pageId=1543; Center for Effective Philanthropy, “Foundation Performance Assessment Framework”, http://www.effectivephilanthropy.org/index.php?page=foundation- performance-assessment-framework5. Tamarack Institute for Community Engagement: “Resource Library”, http://tamarackcommunity.ca/g3s4.html; Innovation Network: “Point K Learning Center”, http://www.innonet.org/index.php?section_id=4&content_id=16; Grantmakers for Effective Organizations: “Do Nothing About me Without Me”, http://www.geofunders.org/publications

    ×