M&E for Social Service System Strengthening


Published on

Presented as a Child Status Network webinar led by Kelley McCreery Bunkers and Mari Hickmann.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • MARI – 2 mins Welcome and thank you for joining. Today we are going to present information about ‘social service systems’ and what is meant by that term, and how this system can be monitored and evaluated. This activity is lead by a PEPFAR-supported project, MEASURE Evaluation, a consortium of six organizations lead by the University of North Carolina. My name is Mari Hickmann, M&E advisor at Futures Group International and OVC M&E technical advisor for MEASURE Evaluation, with experience working with livelihood and OVC-related programs in Asia and Sub-Saharan Africa. I’m joined as a presenter by Kelley McCreery Bunkers, a child protection expert with more than twenty years of professional experience in E. Europe, Latin America and E. Africa. First, Kelley and I will be making a 20-30 minute presentation. We will then open any remaining time to questions, comments and discussion. And you will see that we do have specific questions we’d like to ask you all towards the end. If there are urgent questions, please feel free to type them into the chat box. Otherwise, please hold until the end. Stakeholder options Service provider Government Donor Other Are you based in: Sub-Saharan Africa Asia Latin America United States Other
  • MARI – 2 mins MEASURE Evaluation is supporting PEPFAR in the effort to develop a M & E Framework for the Social Service System.  We have recently initiated this work and our first phase includes  a thorough literature review of peer reviewed and non-peer reviewed documentation. We have gathered literature from the child protection and social protection arenas, as well as examples from the US child welfare system and systems strengthening efforts in Eastern Europe.  At the end of this review, a core set of indicators will be presented for discussion.     The next phase of the work will include finalization of the M & E Framework and a handbook to support design and implementation of the framework in the different countries where PEPFAR and their partners are engaged in systems strengthening work.  We are trying to gather as much information as possible related to existing M & E frameworks, related indicators and promising practices.  We also recognize that there might be gaps and want to highlight these when it occurs.   This webinar is the first step in a longer process.  Our goal today is to share our objectives and progress to-date with you, and we are excited about learning from your experience and hope to gather as much existing information as possible to help the new M & E Framework be as informed as possible and relevant to the countries where PEPFAR is engaged.   We envision hosting more webinars throughout the process to ensure active participation and engagement of key stakeholders.   While we will get into the details pertaining to M&E of social service systems, we first want to take a quick step back to set the stage in the broader context of what a social service system is and how it may function.
  • KELLEY – 2 mins In recent years, progress has been made in defining, designing and implementing different system strengthening related interventions. There has been a growing focus on and recognition of systems within the health, child protection and social protection fields.  However, it is apparent that these service systems, especially  those with a child focus, have not always been universally strengthened.  Rather, focus has been placed on certain components of the larger system.  Furthermore,  current system strengthening interventions are not always coordinated and clearly defined, nor are they necessarily sustainable.   In part due to the benefits of many years of health system strengthening, many child protection and/or OVC-related interventions now prioritize attention to social service system strengthening in the support that is being provided to countries.     To clarify, the new PEPFAR Guidance for OVC refers to this as the Social Service System. There are several definitions that have also been utilized to describe this including  social welfare system or social development system.  The term social service system that is utilized by PEPFAR, as will be presented in the next slide,  has as the overarching goal ensuring the welfare and protection of vulnerable children and other populations. 
  • KELLEY – 2 mins The new PEPFAR Guidance for OVC defines the social service system as a combination of both social protection (welfare) and child protection systems.  As illustrated above, the social service system that is envisaged has as it primary purpose to ensure the welfare and protection of children and other vulnerable populations.  This system  includes components that aim to reduce poverty and vulnerability as well as those that protect children from abuse, neglect and exploitation.     There are six main components to that make up a system and all of these components must work together to enhance the effectiveness of the larger system.  These six components reflect much of what the health system has developed in terms of their generic "building blocks"  of the health system. These elements are also similar to core system components identified in much of the child protection systems literature   The six components are linked, multi-dimensional and adaptable, but all are necessary for the larger system to function.  Identifying these six elements of the social service system is and important part of the process and should result in targeted and informed  interventions, strategies and investments. 
  • MARI – 1min To illustrate the system we are talking about, we want to walk through two brief examples of a functional system. These are merely illustrations and we recognize that intervention strategies, response mechanisms, and actors in this system vary from country to country. But our hope is that these illustrations get us thinking about how systems should work, and how we can assess functionality and break downs within a system.
  • MARI – 2 mins In our first example, a child is identified as malnourished and presents signs of stunting. This child’s entry point into the system is through the health extension worker, who refers the caregiver/child to a local health clinic where they qualify for free treatment. At the same time, the family also receives a voucher for access to receive food from the local community food bank. The health extension worker continues to monitor this household. In doing so, they determine the family could benefit from additional social services. They refer a social welfare officer to the household and the family is referred to local parenting classes, as well as the office that oversees cash transfers. Over time, household wellbeing improves and the child is back at a healthy weight. The family is able to provide for their children, with support from various programs.
  • KELLEY – 2 mins A neighbor suspects that the child next door is being abused. She reports it to the local authorities. The local authorities document the concern and identify a social worker to make house visit.  The social work goes to the home, meets with the mother and then child and completes an assessment of the child and family situation.  The SW does not detect abuse rather identifies that  single mother is overwhelmed, unemployed and in need of support. A case plan is developed, together with the parent and child. It is the clear that the mother needs assistance to get back on her feet and to be able to provide for her child.  Options are explored within the extended family for some temporary care of the child.  No extended family members are willing or able to care for the child so  a  caregiver from the community is identified.  The care situation is approved by local authorities, based on a best interests determination. In the interim, the mother attends a parenting class and is accepted by a local NGO to participate in an IGA scheme. Eventually the child is reintegrated in the family and the social worker monitors the case until it is determined that the child's safety and well-being are secured.  
  • MARI – 2 mins While there are many break downs that may occur to this system, following the six components described earlier, some examples of the challenges that might occur with the first case include the following: First, coordination between health extension workers and social welfare officers may not occur or be as successful as we’d like to see Second, the existing work force may not be able to handle all cases in a community and are overburdened Finally, financing for cash transfers, vouchers and subsidies may be insufficient to support those in need and as a result, families cannot accessible key services or are put on a waitlist
  • KELLEY – 2 mins Some examples of the challenges that might occur with the second case include the following: First, local government authorities mandated with child protection might not be present at the community or town level, reflecting a challenge with the leadership and governance component of the system.   Similarly, data collection systems around children placed in temporary care (either with extended family, community members or foster care) might not be collected.   Finally, service delivery mechanisms and tools used to ensure participation of the family and child (depending upon the age and capabilities of the child) and to make an informed decision regarding best interests determination might not be defined, practiced or required.  
  • MARI The mounting attention to social service system strengthening is the wave of the future and welcomed by stakeholders around the globe. Yet these initiatives will not be sustainable without a monitoring strategy that enables decision-makers to assess the existing system, monitor improvements and advocate for targeted system strengthening needs. The M&E framework should include key indicators to monitor core components of the system – and provide information on where the system may need improvement. This includes looking at input, output, process and outcome indicators.
  • MARI There are six main components to that make up a system and all of these components must work together to enhance the effectiveness of the larger system.  These six components reflect much of what the health system has developed in terms of their generic "building blocks"  of the health system. These elements are also similar to core system components identified in much of the child protection systems literature   The six components are linked, multi-dimensional and adaptable, but all are necessary for the larger system to function.  Identifying these six elements of the social service system is and important part of the process and should result in targeted and informed  interventions, strategies and investments.  The descriptions we use here are summaries from the PEPFAR OVC Guidance and in the context of this webinar are for illustrative purposes only. These brief descriptions are meant for you to consider monitoring and evaluation of this component and how it should be done, or maybe is already being done within one of your programs or within an entity you work with. For each of the 6 components we will provide a brief description, as well as some of our current thinking and examples of SOME indicators. These are not meant to be final indicators, nor are we suggesting these are comprehensive. But moreso to give us an idea of what we’re looking at and points for discussion.
  • KELLEY Would include governments with clear mandates, visions and strategies in place. Organizational structures at all levels, clear understanding of roles and responsibilities of all key actors involved, effective lines of communication across departments and sectors. Examples of possible indicators: A National Strategy for Addressing Child Vulnerability is developed and approved Ministry x has approved/updated job descriptions for all key staff involved in social service system
  • MARI
  • MARI
  • MARI
  • MARI – 1 min Our intention is to develop an M&E framework that can monitor the strength of a country’s social service system. Collection and use of such data will provide information for country decision-makers to inform program and policy development. This framework is also intended to provide a foundation for global monitoring of social service system strengthening efforts.
  • KELLEY – 2 mins We hope that through this process we will be able to develop a M & E framework for the larger social service system.  This should not be considered as a tool to evaluate or assess a specific intervention of a project.  What we hope it will be able to do eventually is to help guide and inform the development of M & E frameworks, indicators and data collection methods of all key stakeholders involved in this important work so that the data flows from and is reflective of the many different stakeholders involved in the social service system and their different interventions. 
  • KELLEY – 3 mins In initiating this work we do know that there are significant, albeit nascent, work being done on systems within the child protection and social protection arenas.  We are reviewing the impressive work that has been done in those fields and are utilizing indicators and promising practices where available.   We are also reviewing the US child welfare system and looking towards that model for possible indicators and guidance.  The extensive work on strengthening child protection systems in Eastern Europe over the past two decades is also providing examples of what we might include in this new framework.     With all of these were are also very aware of how they will need to be contextualized to the countries and cultures present in the countries where PEPFAR works.   
  • MARI – 2 mins
  • MARI – 1 min
  • M&E for Social Service System Strengthening

    1. 1. M&E for Social Service Systems Webinar October 17th 2012By Kelley McCreery Bunkers & Mari Hickmann
    2. 2. Developing M&E for Social Service Systems Phase I: literature reviews, stakeholder inputs, information gathering Phase II: suggest a core set of indicators (based on‘best-practices,’ ‘promising practices’ or informed recommendations) Phase III: final M&E framework and handbook
    3. 3. Evolution of Social Service Systems Provision of material goods to system investments 2010 Social Welfare Workforce Conference in Capetown 2011 and 2012 Child Protection System Conferences (Nairobi & Dakar & India) Growing interface between child protection and social protection systems
    5. 5. Example #1
    6. 6. Example #2
    7. 7. System Break DownsUsing the illustrations from previous slides, this is whatmight occur when the system is not functioning well:Example 1
    8. 8. System Break DownsExample 2
    9. 9. How do we know the strength of the system?“the strength of a system depends on its ability to carryout its functions…which in turn is influenced by thecapacity and interactions of actors across systemlevels”- PEPFAR OVC Guidance
    10. 10. M&E of Key System Components1. Leadership & governance2. Well-performing workforce3. Adequate financing4. Information management / accountability5. Coordination & networking6. Service models and service delivery
    11. 11. 1. Leadership & Governance Clear mandates, visions and strategies Organizational structures, roles & responsibilities National policy / policy implementationExamples of possible indicators: National Strategy for Addressing Child Vulnerability is developed and approved Is there a national Social/Child Protection policy? Sources: Maestral, 2010; Save the Children,2008
    12. 12. 2. Well-Performing Workforce Human resource data and staffing plans Capacity building / training opportunities Occupational standards and codes of conductExamples of possible indicators: National human resource assessment been completed Clear titles, competencies and functions are defined for the different cadres of the workforce # of social service workers / 10,000 population Sources: Maestral, 2010; WHO, 2010
    13. 13. 3. Adequate Financing Costs and benefits of services National budget allocation Financial management systemsExamples of possible indicators: % of total national and local government spending allocated to social services for children Recognized cost/service (e.g., alternative care) Sources: Maestral, 2010; Save the Children,2008
    14. 14. 4. Information Management / Accountability Data management systems Clear mandates to collect specific data Usage of data to inform policy and programmingExamples of possible indicators: Clear monitoring and evaluation functions of key staff involved in child protection at the national level Data utilized to develop in X laws, policies, strategies Sources: Save the Children, 2008; Maestral, 2010; US DHHS, 2006
    15. 15. 5. Coordination & Networking Stakeholder mapping Coordination mechanisms Collaborative agreements with different actorsExamples of possible indicators include: Terms of Reference for National Technical Working Group is developed and approved # of actors engaged in system disaggregated by type of function/service offered/location, etc. Sources: US DHHS, 2006; Maestral, 2010
    16. 16. 6. Service Models / Delivery Mechanisms Service delivery strategy Clear case management practices Promotion, prevention and response servicesExamples of possible indicators could include: Quality Service Standards exist Case review system is in place at all levels Data on # of children outside of parental care Sources: US DHHS, 2006; BCN & UNICEF, 2009
    17. 17. The purpose of this M&E framework is to… Guide countries in designing/implementing social service system strengthening efforts Provide a framework to monitor system improvements Deliver strong evidence to support national strategies and advocacy Provide a foundation for global monitoring of social service system strengthening efforts
    18. 18. But it is not… Meant to be used by service providers to evaluate the impact a targeted intervention(s) Meant to provide qualitative indicators (which are recommended to complement quantitative)
    19. 19. What we know (or think we know)M&E of Child/Social Protection Systems is nascent  But there is helpful documentation, info and tools to supportSemi-relevant examples from the U.S. child welfare systemand systems development from Eastern Europe  But these are not perfect, and not necessarily applicable to all countries we’re working inApplicable & relevant lessons from health systemstrengthening  But health ≠ equal child social services
    20. 20. What we would like from you Overall reactions and feedback Are there examples of social welfare system assessments (outside the U.S.)? Are there examples of Ministries/Governments that monitor or evaluate (parts of) their social service system? Do any of the programs that you work in or with use indicators that monitor any of the 6 system components?
    21. 21. How you can contribute1. Raise you hand to speak2. Type comments / questions into the chat box3. Join the discussion online at: http://childstatus.net/social-service-systems/4. Email us: • Mari Hickmann: mhickmann@futuresgroup.com • Kelley McCreery Bunkers: kelleybunkers@gmail.com
    22. 22. THANK YOU!MEASURE Evaluation is a MEASURE project funded by the U.S.Agency for International Development and implemented by theCarolina Population Center at the University of North Carolina atChapel Hill in partnership with Futures Group International, ICF Macro,John Snow, Inc., Management Sciences for Health, and TulaneUniversity. Views expressed in this presentation do not necessarilyreflect the views of USAID or the U.S. Government. MEASUREEvaluation is the USAID Global Health Bureaus primary vehicle forsupporting improvements in monitoring and evaluation in population,health and nutrition worldwide. Visit us online at http://www.cpc.unc.edu/measure.