The document summarizes different approaches to evaluation in the third sector over time. It discusses the evolution of policy emphasis from efficiency in the early 1990s to effectiveness in the mid-1990s to performance and value for money currently. Key terms like monitoring, evaluation, outputs, outcomes, indicators, and intermediate outcomes are defined. Steps to self-evaluation and using tools like logic models and the Outcomes Star are presented. The importance of evaluation for both accountability and learning is highlighted.

1. Methodologies and approaches to evaluation in the third sector International conference on evaluation, Barcelona 3 November 2011 Dr Jean Ellis Charities Evaluation Services

2. The policy waves Early 1990s – more business-like sector and efficiency focus Mid 1990s – focus on results and effectiveness From 2004 – emphasis on performance and value for money Current situation – emphasis on savings and social value

3. Some definitions Monitoring The routine, systematic collection of information for the purpose of checking progress against project plans Evaluation Using monitoring and other information to make judgements and give value

4. Evaluation Evaluation aims to answer agreed questions and to make a judgement against specific criteria. Like other research, for a good evaluation, data must be collected and analysed systematically, and its interpretation considered carefully . CES (2009) Practical Monitoring and Evaluation

5. The first policy wave Early 1990s – drive for a more business-like sector and efficiency focus

6. The purposes of evaluation Evaluation for accountability – to demonstrate achievement and compliance Evaluation for learning – to share findings, improve, replicate and innovate

7. Monitoring and evaluation: is good for you is good for users meets external demands can make your organisation better Why do it?

8. Steps to self-evaluation 1- Clarify aims and outcomes, objectives and outputs 2- Set performance indicators; choose tools 5-Use findings 3- Collect information 4- Analyse and interpret FIVE STEP PLAN

9. CES planning triangle © Overall aim Objectives This is what we do Specific aims This is why we do it Outputs Outcomes

10. The Family Centre To provide opportunities for play, drama and dancing To provide workshops, information and advice on parenting To organise outings for children and their parents To improve the lives of the children who use the centre To increase children’s self-esteem To enable parents to give each other practical support Objectives Specific aims Overall aim To run a drop-in centre with a play area To improve the parenting knowledge and skills of the parents who use the centre

11. Second policy wave Mid-1990s – focus on results and effectiveness

12. More definitions Outputs are an organisation’s detailed activities, services and products – what you do Outcomes are all the changes, benefits, learning or other effects that happen as a result of an organisation’s activities – what you change

13. Aims, outcomes and indicators Aims – the broad changes you plan to achieve  Outcomes – the specific changes for individuals, families, communities, organisations and policy  Indicators – well-defined and measurable information that will show change

14. Intermediate outcomes There are often smaller changes that need to happen before the final, desired outcome can be reached. These steps along the way are known as intermediate outcomes. Families engage with befriending service  Greater awareness within the family of later life conditions  Families feel supported, less isolated  Increased communication and coping skills  Improved family relationships and dynamics

15. The Outcomes Star © Triangle Consulting 2011

16. Underpinned by a model of change © Triangle Consulting 2011

17. Third policy wave From 2004 – emphasis on performance and value for money

18. Drowning in data Too much time collecting and reporting information Data not always analysed or used Costly in time and resources But monitoring and evaluation IT systems could help greatly.

19. Current policy wave The current situation – emphasis on savings to the public purse and social value .

20. Using Social Return on Investment Fab Pad supports young homeless people to sustain new tenancies. The SROI evaluation carried out on Fab Pad revealed that for every £1 invested by the government in support, £8.38 of social return was derived in reduced health care costs, reduced welfare benefits expenditure and reduced costs of repeat homelessness.

21. Evaluation for learning What worked? What parts worked? Who did they work for? In what contexts did they work? What else happened? Were there any negative consequences? Did the results last?

22. Accountability or Learning? Closed enquiry Proving success Showing cause and effect Used for reporting and marketing Open enquiry Valuing failure as well Understanding context Used for improvement and change.

23. Theory of change It makes clear all the assumptions about how change will happen. Activities Short-term change Steps along the way Longer-term change

24. Challenges and opportunities Managing data Getting skilled Finding meaning Methods Innovation Using findings New technology

25. A winning charity The Leeds Survivor Led Crisis Service: We are passionate about monitoring and evaluation as we believe it is the absolute right of our visitors and callers to know that we scrutinise our work in order to ensure that we are providing an excellent service which meets their needs.

26. CES contacts: www.ces-vol.org.uk 020 7713 5722 [email_address] Jean Ellis : [email_address] 02070789386