Policy imperatives for systems-oriented approaches to scaling up


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • These slides summarize findings from a review that Rebecka did on scaling up research for class last year.
  • Concerns cited in the literature:
    Point 1: Practices spread if interpretive process renders them salient, familiar, compelling – e.g. if the idea of the SDM makes sense culturally.
    Point 3: Researchers tend to underemphasize rejection/discontinuance/re-invention of innovations because they are frequently part of the scale-up process.
  • A big program challenge is defining scale up and its attributes. It has become a jargon term with many meanings.
    People often say that a FP method is ‘scaled up’ once it is found in the MOH’s FP norms and procedures and when providers have been trained in offering the method. But full scale integration of a new method (or any other kind of new service) touches on many systems elements in order to be sustained. As the slide shows, systems and services are interlinked. Political support and technical leadership provide the forward momentum.
    When you think of what is the innovation – many elements go into the package that makes up the innovation. It is not just the technology being introduced.
  • For program management and decision making
    Also for understanding SU processes
  • Yin, R. (1994). Case study research: Design and methods (2nd ed.). Thousand Oaks, CA: Sage Publishing.
  • As opposed to the null hypothesis
    –Scale-up efforts which focus primarily on integration of SDM into norms and training will not lead to sustainable quality SDM services.
  • How to study scaling up? It is a very complex process with a wide range of outcomes – some which we can predict, others not.
    Case study research can help us focus our study and ensure that we collect sufficient information to answer questions about the process of scaling up and its outcomes.
  • Operationalizing scale up indicators – so that it could be evaluated/researched
  • I would say that based on our hypothesis and theory of change – IRH inputs are focused on Resource Org. Activities, and helping our partners do the user org. activities.
  • This slide is continuation of our model (principles). And we are trying to integrate these aspects into both our planning and M&E and research
  • First two outcomes are fairly standard.
    With scaling up goals, we are also looking at outcomes that relate to systems integration and resource mobilization.
    The latter is critical in terms of achieving sustainability – when those who are adding the SDM use their own resources to sustain the introduction.
  • Some process questions – to help understand and document important factors in scale up processes
  • Mix of qualitative and quantitative data.
  • Overview from different countries of the different kind of data being collected to inform scale up in the different countries.
    Multiple sources will provide ways to validate information as well as inform program planning and monitoring scale up progress.
  • This are indicators collected by our biannual reporting. I put Mali as an example
    Shows year 2 of a five year plan.
    Shows that scale up is not really measured by yes/no.
  • Information from policy makers/program managers from stakeholder interviews in Guatemala
    Teasing out at central and other levels factors influencing scale up of the SDM. Questions reflect elements of the scaling up model of Expandnet – looking at system capability, political factors, resource factors.
  • Scale up barriers/successes seen at level of service delivery – provider interviews and facility assessments – Rwanda example
  • Another info source from which to monitor / evaluate / inform strategies of scale up is household surveys – of women and men.
    Findings of women respondents in Mali capturing attitudes towards the SDM.
    Also look at traditional FP indicators such as method mix and SDM contribution. Really important info for policy makers and program managers since HMIS is not yet capturing SDM users.
  • Added one inductive methodology to capture significant changes as recounted by policy makers and program planners, those providing and supervising the integration of the SDM into FP services.
    Story-telling – asking what is the most significant change you have seen in the last year in terms of rolling out the SDM, and why do you think it is the most sigfnicant.
  • Shifts in partners, donors, intervention areas, unanticipated events result in issues with design/data collection
  • Policy imperatives for systems-oriented approaches to scaling up

    1. 1. Policy imperatives for systems-oriented approaches to scaling up: Case example of taking the SDM to national scale Susan Igras, Rebecka Lundgren, Marie Mukabatsinda, Arsène Binanga, Foufa Touré International FP Conference, Uganda, Nov 2009
    2. 2. Scale Up Research: Where are We? • Extensive research on attributes of innovation, patterns of adoption, use of opinion leaders • Inadequate understanding of processes by which innovations are implemented and sustained • Information rarely gathered (considered part of routine program operations)
    3. 3. Scale up Research: Some Concerns • Tendency to focus on individual rather than systems level • Typically ignores interpretive process underlying adoption • Pro-innovation bias of researchers (who are frequently part of scale up process) • Need in-depth mixed-method studies to provide picture of process and impact • Not theory-driven
    4. 4. FAM Project Scaling up the Standard Days Method • DRC • Guatemala • India (Jharkhand) • Mali • Rwanda
    5. 5. Scaling up ≠ Training + FAM in norms •Norms & procedures •Training curricula •Supervision •HIS •Procurement & distribution •Budget lines •Provider training •Availability of CycleBeads •Supportive supervision •IEC: new method as part of FP mix
    6. 6. FAM Project Research • 5 year prospective, multi- site, comparative study of process and outcomes of scaling up a FP innovation, the Standard Days Method • Uses ExpandNet model for planning, monitoring and research
    7. 7. Why Study FAM Scale up? • Guide the scale-up process of FAM integration • Maintain momentum and accountability of multiple stakeholders/actors • Assess whether scale up is achieved in terms of outcome and impact • Describe the process of scaling up • Understand better determinants of success and failure • Contribute to program knowledge base on scaling up
    8. 8. Case study research (Yin, 1994) an empirical inquiry that investigates a contemporary phenomenon within its real-life context; when the boundaries between phenomenon and context are not clearly evident; and in which multiple sources of evidence are used.
    9. 9. Hypothesis -Applying the systems analysis framework and scaling-up principles articulated in the ExpandNet framework -to scaling up the SDM within existing programs and services -will lead to more sustainable, quality SDM services.
    10. 10. Case study on scaling up SDM • Brings us to an understanding of a complex issue or object • Can extend experience or add strength to what is already known through previous research • Emphasizes detailed contextual analysis of a limited number of events or conditions and their relationships
    11. 11. Defining the case study parameters • Logic model • Relational model • Performance indicators
    12. 12. Defining Success in Scale Up
    13. 13. Strategic Choices for FAM Scale Up Lead to Selection of Inputs User Organizations- Activities • Training and supervision • Community awareness raising (IE&C) • Multi-systems integration Resource Organizations- Activities • Advocacy • Building leadership • Capacity building in systems analysis • Transferring skills and ownership
    14. 14. ExpandNet Scale up Principles Influence Strategic Choices • Participatory • Systems approach • Evidence based • Quality assurance • Sustainability goal • Rights oriented and equitable (gender, vulnerability, and client)
    15. 15. Research Questions: Scale-up Outcomes Client What is the experience of women and men with the SDM when scaled-up? (Knowledge, attitudes and use of the SDM) Service offering Is SDM offered correctly by providers? How does SDM introduction influence quality, availability and use of overall family planning services? System integration To what extent has SDM been integrated into training, IEC, procurement and distribution, and HMIS? Is included in norms, protocols and guidelines? Resource mobilization What is the level of non-USAID resources dedicated to FAM?
    16. 16. Research Questions – Scale-up Process Resource team To what extent do user organizations assume the roles, responsibilities and ownership of the resource team during the scale-up process? Advocacy/ Dissemination What is the role of champions of the SDM? Organizational choices To what extent has the SDM been offered outside traditional public sector service delivery?
    17. 17. Data Sources for Scale up Monitoring, Evaluation, and Case Study
    18. 18. Findings from Baseline Research and Process Documentation
    19. 19. Monitoring Performance Benchmarks - Mali Selected Indicators (as of 6/09) No. of resource orgs 11 / 16 SDM included in key policies, norms, protocols 5 / 8 revision needed SDM in pre-service training Initiated Commodities in logistics & procurement systems In CPT SDM in IE&C materials Yes , revision needed SDM in HMIS In process Proportion of SDPs with FAM in method mix 66 % FAM in surveys (DHS) Under discussion Funds leveraged for FAM $75,000 (est’d) Providers trained 4575 / 5394
    20. 20. Hlth/FP Program Mgrs and Policy Makers in Guatemala (n=20) Political commitment to SDM scale up Commitment is there since SDM already integrated (in norms, training occurring, in materials) Political factors in SDM scale up Some stakeholders not convinced a natural method can be modern and effective and that there would be sufficient demand. FBOs, Church, and community based NGO networks strong supporters FAM knowledge/attitudes Almost all were knowledgeable about FAM (but not knowledgeable about efficacy trials and rates) Ability of MOH to manage SDM scale up Was within their mandate. If there is demand, they will support it. Integration of SDM into annual planning / budgeting processes Not yet in planning and budgeting processes centrally or regionally. Felt if SDM ‘proved’ it had high demand, it would be integrated.
    21. 21. Provider Interviews / Facility Assessments in Rwanda (n=155 and n=109) FAM integration into norms, guidelines, policies 2/3 of providers have seen protocols Most not familiar with norms and guidelines (newly introduced in Rw) Status of FAM services 60% of providers have offered SDM (42% in last 3 months) 15% have offered LAM (99% in last 3 months) 70% have been offering SDM between 1-5 years Correctness of SDM info by provider Most providers offer SDM competently, and do not find SDM counseling difficult Service deliv environment Providers only have 4-10 min for counseling on FP – not enough Status of FAM 91% of visited facilities offering FP offered SDM.
    22. 22. Women Survey Respondents-Mali 29 52 39 71 58 38 43 25 26 8 24 3029 23 36 21 18 32 Interfereavec lesrelations sexuelles N'apas d'effets secondaire Acceptable pourhommes Efficace Difficlea utiliser Difficilea comprendre pour parternaire Figure 2. Perception (attitudes) des femmes de la MJF N= 624 Oui Non Nesait pas
    23. 23. Most Significant Change (MSC) Inductive story-based methodology to understand and document… • scale-up processes / outcomes not detected by quantitative monitoring • unanticipated processes/effects of FAM scale up • meanings of scale-up process and outcomes to partners, stakeholders, communities • intangible aspects of FAM scale up (advocacy, leadership, gender equity, informed choice)
    24. 24. Benefits of Scale Up Research • Measures progress • Identifies needed adjustments • Involves high level (all level) stakeholders • Provides evidence for advocacy • Maintains momentum and accountability during a multi-organization/multi-layered process
    25. 25. Scale up Research Challenges • Adapting research to frequently shifting policy and resource environments • Experimental designs difficult to implement • Involving multiple partners in data collection • Producing relevant, timely data for stakeholders with diverse needs •
    26. 26. Methodology: Early Lessons Learned • Operationalization of scale up (indicator selection) critical for planning, monitoring and evaluation • Need to pay attention to interpretation of meanings of indicators – Definitions of scaling up, integration, introduction, availability • Advantages/disadvantages of comparative design • Relational database has facilitated management of diverse info/data
    27. 27. Scale up Process:Early Lessons Learned • Systems approach uses broader lens and uncovers systemic problems – who is responsible? • Planning for scale up important to gain commitment, create accountability, avoid common mistakes • Data conundrum – need new user data to monitor service expansion and justify value of investment, SDM users not yet counted in HMIS • New skills and attitudes needed for effective scale up, eg, political and systems analysis, transferring responsibilities for technical / other support
    28. 28. If ultimate goal is sustained, equitable access to new product/service at scale • Policy makers, donors, program managers need better understanding of complexity of scaling up and resource investments • Introduction is a systems-shifting process influenced by factors both inside and outside of immediate FP program environment • Planning for and monitoring scale up can create MOH and partner accountability, avoid crisis management, and help ensure that what is scaled up is sustained