Introduction to the Improve It Framework -Effectiveness:Improving and Proving It ConferenceEdinburgh November 2012Ben Jack...
Context
Consortium Effectiveness Programme…supports UK NGOs in improving how they measure, manage anddemonstrate their contributio...
Consortium Effectiveness Programme:Key elements    • NIDOSEffectiveness Tool /    Bond Organisational Health Check    • Va...
Contributors to the programmeConsortium partners:                                bond.org.uk   5
How the strands of programme fit together                          Development                          effectiveness     ...
Improve ItFramework
The Improve it FrameworkWhat is it?    •   A tool for improving how UK NGOs measure,        learn from and demonstrate out...
Improve it Framework: what it is and is not                 It is not…                                      It is…•     …a...
Where the Improve it Framework can help inyour work                                                Focus of Improve it Fra...
In practice
Improve itFramework  Capacitydevelopment
Domains of Change Framework for Education                                   Outcomes                                   a)...
Domains of Change Framework for Education                                   Outcomes                                   a)...
Domains of Change Framework for Education                                   Outcomes                                   a)...
School management committees exist and manage schools effectively                Indicators                               ...
Timelines
What to look forward to•   Organisational Health Check pilot starts (Summer 2012)•   Launch of pilot Evidence Principles (...
Thank youContact:Ben JacksonChief Executivebjackson@bond.org.ukRegent’s Wharf • 8 All Saints Street • London • N1 9RL • Un...
Organisational Health CheckWhat is it?•       An online self-assessment that:    •    Identifies strengths and weaknesses ...
Value for Money (VfM)What are we doing?•       Providing common tools and frameworks        for NGOs to navigate the VfM a...
NGO Evidence Principles (pilot to May 2013)1.   Voice and Inclusion: We listen to beneficiaries’ perspectives on the     e...
NGO Evidence Principles – the checklistA NGO checklist for assessing the quality of evidence (DRAFT)Piece of evidence bein...
NGO Evidence Principles – the results  •   A quality assurance      framework                        Voice & Inclusion    ...
Effectiveness: Improve It Framework - Ben Jackson, Bond
Effectiveness: Improve It Framework - Ben Jackson, Bond
Effectiveness: Improve It Framework - Ben Jackson, Bond
Effectiveness: Improve It Framework - Ben Jackson, Bond
Upcoming SlideShare
Loading in …5
×

Effectiveness: Improve It Framework - Ben Jackson, Bond

1,192 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,192
On SlideShare
0
From Embeds
0
Number of Embeds
171
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Effectiveness: Improve It Framework - Ben Jackson, Bond

  1. 1. Introduction to the Improve It Framework -Effectiveness:Improving and Proving It ConferenceEdinburgh November 2012Ben Jackson Bjackson@bond.org.ukChief Executive, Bond bond.org.uk
  2. 2. Context
  3. 3. Consortium Effectiveness Programme…supports UK NGOs in improving how they measure, manage anddemonstrate their contribution to social change1. Developing agreement and supporting implementation of a sector wide framework of common outcome indicators and data collection tools (Improve it Framework); and online organisational self- assessment and resource portal (Organisational Health Check; NIDOS Effectiveness Tool); and agreed quality standards regarding NGO evidence.2. Building knowledge and skills to support members to measure and manage effectiveness and value for money3. Creating an enabling environment that encourages and supports organisations to deliver improvements in their effectiveness bond.org.uk 3
  4. 4. Consortium Effectiveness Programme:Key elements • NIDOSEffectiveness Tool / Bond Organisational Health Check • Value for Money • NGO Evidence Principles • Improve it Framework bond.org.uk 4
  5. 5. Contributors to the programmeConsortium partners: bond.org.uk 5
  6. 6. How the strands of programme fit together Development effectiveness Robust, Appropriate practical approaches to approaches to managing cost measuring efficiencies & effectiveness effectiveness Strong systems, culture, leadership etc Bond Organisational Health Check/ NIDOS Effectiveness Tool bond.org.uk 6
  7. 7. Improve ItFramework
  8. 8. The Improve it FrameworkWhat is it? • A tool for improving how UK NGOs measure, learn from and demonstrate outcomes and impact • A platform for sharing information on outcomes, indicators and data collection toolsWhy are we developing it?• Developing robust methods for measuring effectiveness is a collective challenge for UK NGOs• UK NGOs need to start telling a more robust story at sector level as well as at organisational level about their added value bond.org.uk 8
  9. 9. Improve it Framework: what it is and is not It is not… It is…• …about creating a single way of • …about providing a collective resource assessing effectiveness –it’s about that UK NGOs can draw on when encouraging greater harmonisation developing their own context specific and consistency where appropriate M&E frameworks• …going to offer an off the shelf • …about developing shared answer to measuring effectiveness, approaches where we can but it will provide a common starting • ….about providing organisations with point the tools to be able to tell a more• …going to produce an encyclopaedia robust story of how they are of indicators and tools. There will be contributing to change an element of prioritisation. • …going to be practical & simple• …an organisation specific tool; • …going to be dynamic & will continue seeing 60% of your organisation in it to evolve is good enough bond.org.uk 9
  10. 10. Where the Improve it Framework can help inyour work Focus of Improve it Framework 1 2 3 4 5 6 Identify who Identify the Collect,Identify the Articulate you will work Identify what evidence analyze andproblem & your vision with & you want to that you will drawunderlying of change influence & change collect and conclusions causes how how for the data Outcomes Indicators & Principles 7 8 toolsReflect andlearn from Communicate the data results bond.org.uk
  11. 11. In practice
  12. 12. Improve itFramework Capacitydevelopment
  13. 13. Domains of Change Framework for Education Outcomes a) Laws, policies and practices b) Adequate resources are allocated c) Performance in are in place that support quality and adequate teachers recruited and education is monitored education that is accessible for trained for the provision of quality at a national and local all children education for all level OutcomesOutcomes 2 a) School management Mothers, fathers and othera) Governments and committees exist and managecaregivers support all their other mandated bodies schools effectivelychildren to access education ensure all children b) School managers access,b) Communities actively support access quality manage, and deploy resourcesand participate in children’s education in an effective and transparenteducation way 6 3c) CSOs support and advocate for c) Teachers, resources andimproved education for all infrastructure are in place to Communities actively Schools are well support education support the education managed, safe and well of all children d) Schools are safe and resourced healthy environments for girls 1) All children and boys access quality education where Outcomes they learn useful knowledge and a) Teaching is high quality, skills child-centred and child-Outcomes friendlya) Children participate in a b) Teachers deliver a fullmeaningful way in decisions that 5 4 allocation of lessonsaffect their education c) Teachers are trained andb) Children participate in their All children participate All children receive supported to deliver highown and others’ learning in their education and their full allocation of quality teaching in school life high quality teaching d) The curriculum is relevantc) Children have a positive and appropriateattitude towards education e) Teaching supports equality and empowerment
  14. 14. Domains of Change Framework for Education Outcomes a) Laws, policies and practices b) Adequate resources are allocated c) Performance in are in place that support quality and adequate teachers recruited and education is monitored education that is accessible for trained for the provision of quality at a national and local all children education for all level OutcomesOutcomes 2 a) School management Mothers, fathers and othera) Governments and committees exist and managecaregivers support all their other mandated bodies schools effectivelychildren to access education ensure all children b) School managers access,b) Communities actively support access quality manage, and deploy resourcesand participate in children’s education in an effective and transparenteducation way 6 3c) CSOs support and advocate for c) Teachers, resources andimproved education for all infrastructure are in place to Communities actively Schools are well support education support the education managed, safe and well of all children d) Schools are safe and resourced healthy environments for girls 1) All children and boys access quality education where Outcomes they learn useful knowledge and a) Teaching is high quality, skills child-centred and child-Outcomes friendlya) Children participate in a b) Teachers deliver a fullmeaningful way in decisions that 5 4 allocation of lessonsaffect their education c) Teachers are trained andb) Children participate in their All children participate All children receive supported to deliver highown and others’ learning in their education and their full allocation of quality teaching in school life high quality teaching d) The curriculum is relevantc) Children have a positive and appropriateattitude towards education e) Teaching supports equality and empowerment
  15. 15. Domains of Change Framework for Education Outcomes a) Laws, policies and practices b) Adequate resources are allocated c) Performance in are in place that support quality and adequate teachers recruited and education is monitored education that is accessible for trained for the provision of quality at a national and local all children education for all level OutcomesOutcomes 2 a) School management Mothers, fathers and othera) Governments and committees exist and managecaregivers support all their other mandated bodies schools effectivelychildren to access education ensure all children b) School managers access,b) Communities actively support access quality manage, and deploy resourcesand participate in children’s education in an effective and transparenteducation way 6 3c) CSOs support and advocate for c) Teachers, resources andimproved education for all infrastructure are in place to Communities actively Schools are well support education support the education managed, safe and well of all children d) Schools are safe and resourced healthy environments for girls 1) All children and boys access quality education where Outcomes they learn useful knowledge and a) Teaching is high quality, skills child-centred and child-Outcomes friendlya) Children participate in a b) Teachers deliver a fullmeaningful way in decisions that 5 4 allocation of lessonsaffect their education c) Teachers are trained andb) Children participate in their All children participate All children receive supported to deliver highown and others’ learning in their education and their full allocation of quality teaching in school life high quality teaching d) The curriculum is relevantc) Children have a positive and appropriateattitude towards education e) Teaching supports equality and empowerment
  16. 16. School management committees exist and manage schools effectively Indicators Tools• # and % of schools with a Parent Teacher The Right to Education indicators and form Association (PTA) for section 9: right to transparent and accountable schools.• Level of activity of the Parent Teacher The Plan UK child friendly tool Section 1: Association (PTA) School management• # and % of schools with a School The VSO quality scale for education services management committee (SMC) quality areas on education management and community engagement• Level of activity of the School management committee (SMC) bond.org.uk 16
  17. 17. Timelines
  18. 18. What to look forward to• Organisational Health Check pilot starts (Summer 2012)• Launch of pilot Evidence Principles (October 2012)• Beta version of the Improve it Framework (December 2012)• Comic Relief, Big Lottery Fund, DFID, Bond research into the cost of measuring effectiveness (Spring 2013)• Guide to demystifying theories of change (Spring 2013) bond.org.uk 22
  19. 19. Thank youContact:Ben JacksonChief Executivebjackson@bond.org.ukRegent’s Wharf • 8 All Saints Street • London • N1 9RL • United Kingdom+44 (0)20 7837 8344 • bond@bond.org.uk • bond.org.uk bond.org.uk 23
  20. 20. Organisational Health CheckWhat is it?• An online self-assessment that: • Identifies strengths and weaknesses in organisational capacity / functionality • Facilitates benchmarking with peers • Sign posts to other initiatives, tools, training and supportWhy are we developing it?• Strong systems, leadership etc are key building blocks for delivering good development outcomes• A common framework can help facilitate structured peer learning and the sharing of best practice• An online self-assessment enables bond to provide more tailored capacity support bond.org.uk 24
  21. 21. Value for Money (VfM)What are we doing?• Providing common tools and frameworks for NGOs to navigate the VfM agenda• Bringing NGOs together to share emerging experience• Ensuring constructive dialogue with DfID Why are we doing it? • At the heart of the VfM agenda are important questions about how UK NGOs use their resources • VfM is a part of the wider results agenda • Want to ensure VfM remains appropriate and proportionate to NGO operations and programmes bond.org.uk 25
  22. 22. NGO Evidence Principles (pilot to May 2013)1. Voice and Inclusion: We listen to beneficiaries’ perspectives on the effects of the intervention, and can identify who has been affected and how.2. Appropriateness: We use methods for data collection and analysis that are justifiable given the nature of the intervention and purpose of the assessment3. Triangulation: We make conclusions about the intervention’s effects by using a mix of methods, data sources, and perspectives4. Contribution: We show how change happened, explore a range of explanations for why it happened, test our theory of how we thought it would happen, and explain how we contributed5. Transparency: We are open about the data sources and methods used, the results achieved, and the strengths and limitations of the evidence bond.org.uk 26
  23. 23. NGO Evidence Principles – the checklistA NGO checklist for assessing the quality of evidence (DRAFT)Piece of evidence being assessed: Name of assessor: Principle Criteria 1 2 3 4 NA Comments / evidence 1a. Are the perspectives of beneficiaries included in the1) Voice and Inclusion evidence? We listen to 1b. Are the perspectives of the most excluded and beneficiaries’ marginalised groups included in the evidence? perspectives on the effects of the 1c. Are the findings disaggregated according to sex, intervention, and can disability and other relevant social differences?identify who has been affected and how 1d. Do beneficiaries play an active role in designing the assessment process and/or analysing data? Score for voice and inclusion (average of criteria scores): 2a. Are the methods used to collect data appropriate given 2) Appropriateness the nature of the intervention and the purpose of the assessment?We use methods that 2b. Is the size and composition of the group from whichare justifiable given data is collected appropriate given the nature of the the nature of the intervention and purpose of the assessment? intervention and 2c. Is the composition of the team collecting and analysing purpose of the the data appropriate given the nature of the intervention assessment and purpose of the assessment? 2d. Are the methods used to analyse the data appropriate given the nature of the intervention and purpose of the assessment? Score for voice and inclusion (average of criteria scores): bond.org.uk
  24. 24. NGO Evidence Principles – the results • A quality assurance framework Voice & Inclusion Appropriateness 13/16 7/16 • A kite mark / seal of approval Transparency Triangula on 9/16 12/16 Contribu on • A tool for identifying skill 4/16 gaps and building capacity bond.org.uk 28

×