Download to read offline
Uploaded byAustralian Federation of AIDS Organisations
Understanding what works and why in peer and community based programs for HIV and HCV
This document discusses using systems approaches to better understand peer-based programs for HIV and HCV. It summarizes work done with various organizations representing people who use drugs, gay men, people living with HIV, sex workers, and others. Systems approaches were used to develop more sophisticated theories of how peer-based programs work and influence communities. System dynamics maps showed how interventions engage with communities. Key functions and draft indicators were identified to demonstrate influence and help programs evaluate their work and influence on communities and policies. The document provides an overview of the Understanding What Works & Why (W3) project which aims to help answer questions about program influence and effectiveness using systems approaches.
More Related Content
Similar to Understanding what works and why in peer and community based programs for HIV and HCV
More from Australian Federation of AIDS Organisations
Understanding what works and why in peer and community based programs for HIV and HCV
- 1. Understanding what worksand why in peer and community based programs for HIV and HCV Dr Graham Brown and Daniel Reeders www.w3project.org.au latrobe.edu.au/arcshsMelbourne, Australia
- 2. Peer Based Programs •Peer programs underpin the partnership approach in HIV and HCV • Programs are inextricably linked to constant changes in communities and context • Complex relationship between activities and the broader social impact – difficult to demonstrate • A simple linear approach assuming a stable environment is blind to reality of the programs
- 3. Traditional program logicstruggles to reflect the complex role of PBP in the HIV/HCV response Inputs Activities outcomes Peer-based programs engage with peer interactions, social networks, communities, politics and culture None of which are linear or independent of each other Led us to systems approaches as a tool to elicit what is really going on
- 4. What Works andWhy Using systems approaches to: • develop a more sophisticated understanding of the role of peer based programs and how and why they work; • help community & peer based organisations evaluate what really matters to their work; and • articulate how peer based programs add critical value to HIV and HCV programs and policy • Systems approaches – set of tools to improve understanding of the whole, its components, and in particular the relationships and interactions within and between them.
- 5. What Works andWhy • People who use drugs – WASUA and AIVL • Gay men – VAC and AFAO • PLHIV – NAPWHA and the Poz Action Group • Sex Workers – Scarlet Alliance and members • Funded by the Commonwealth Department of Health
- 6. What Works andWhy • Combine mental models of practitioners to develop a more sophisticated theory of peer based programs and investments • Developed system dynamics maps showing how interventions are embedded in and engage with communities-as-systems • Identifying strategic considerations and key functions within the system maps • Identifying draft indicators to demonstrate influence + a LOT of trust
- 7. Peer network targetedhealth promotion
- 17. 2015 – PracticalIndicators To help answer key questions such as: • Are programs doing what they’d need to do to improve their influence on communities and policy environments (understood as complex systems)? • Are the programs’ understandings of those complex systems close enough to reality to sustain effective action within them? • Can we explain how the program works to stakeholders and justify their investment and trust? • Are program activities, other programs in the sector, and policy enhancing or diminishing each others effectiveness?
- 18. 2015 – PracticalIndicators Flexible use of the functions and indicators • As objectives in strategic planning for program activities • As topic guides for sharing program and organisational knowledge – such as team meetings or planning workshops • In combination - to craft for stakeholders coherent, politically effective narratives of the complex relationships that the program is seeking to influence or activate • To plan how different programs or activities could better work together
- 19. Visit us onthe web at www.w3project.org.au Understanding What Works & Why (W3) is a project of the Australian Research Centre in Sex, Health and Society at La Trobe University. Authored by Dr Graham Brown and Daniel Reeders Funded by the Commonwealth Department of Health Copyright © 2015 La Trobe University
- 20. Function Description Engagement Howwell the program engages with the diversity and dynamism of the affected community, including its different networks and cultures. Learning and adaptation The practices through which the program learns about its environment and adapts its approach according to what it learns. Influence How effectively the program influences its target communities and its policy and funding environment (including indirect influence). Alignment The extent to which different programs in the organisation or sector adapt to achieve ‘value add’ from a coordinated approach.
Editor's Notes
- #3 Australia’s response to HIV and HCV has often celebrated the partnership of affected community, clinicians, research and government. The key role of affected communities being underpinned by concept of ‘peer education’ – although this is rarely defined. Peer-based programs engage with peer interactions, social networks, communities, politics and culture, which are dynamic and adapting. Program and the context are constantly interacting with each other, and as a community driven program they are intended to operate at multiple levels – individual, network, community and structural. However - understanding and demonstrating the complex relationship between an investment in peer based program and achieving improved outcomes is difficult. The inputs and outputs can have a complex and distal relationship A simple linear approach assuming a stable environment is blind to reality of the programs What this often results in is rhetoric about community mobilisation and peer based responses – but contracted, evaluated and managed as information dissemination or generic service provision. Traditional evaluation methods struggle to capture what matters to practitioners and policy-makers about peer approaches. Our hunch is that practitioners have mental models of how peer programs work that may be more sophisticated than published evidence.
- #4 Very few community programs function as a simple input-output-outcomes framework. If we were to really make a shift in the capacity of peer programs to demonstrate their role and influence, we needed to rethink the way we described and evaluated these programs. To do that – we needed a more sophisticated and less linear program theory.
- #5 We applied a systems thinking approach that conceptualises peer based programs, and the communities and policy environments they engage with, as complex adaptive systems. This refers to systems that are constantly interacting, learning, adapting and influencing – and in ways that are not always predictable. The W3 Project is working with CBOs using peer based approaches with a range of different communities and populations: People who use drugs – WASUA and AIVL Gay men – VAC and AFAO PLHIV – NAPWHA and the Poz Action Group Sex Workers – Scarlet Alliance and members Our three goals were: To develop an understanding of how peer based programs work that is more sophisticated than much of the peer reviewed published literature; To help community and peer based organisations identify and evaluate what really matters to their work; To articulate (primarily to funders) how peer based approaches add value to HIV and hepatitis C programs and policy. So why would we want to do that?
- #7 Our approach.. W3 draws on practice based understandings of peer based programs that are often more sophisticated than the individual knowledge focus of much of the published evaluation evidence Bringing together practitioner mental models. Systems thinking is intended to improve the quality of those perceptions of the whole, its parts, and the interactions within and between levels. We have developed ‘system logic’ diagrams showing how interventions are embedded in and engage with communities-as-systems. This process has required a huge amount of trust from our partners – this is breaking new ground.
- #9 As you can see these complex spaghetti and meatball diagrams look very different to the traditional program logic. But these were a tool to that endeavoured to represent the complex reality of each specific program... We certainly had a WTF moment – meaning – what’s the function underpinning these complex maps… We needed to find the commonalities between these four very different program theory diagrams ...so we needed a framework! To answer: What are the key functions any program needs to fulfil in order to be effective and sustainable in a changing context?
- #10 Focus on explaining the clouds – complex systems that are constantly adapting and evolving These two aspects – and how the program interacted with them - were the parts that varied the most across the 4 case trials
- #11 We only include aspects of the program that mattered to our work as evaluation researchers These represent the activities/ Peer skill is the ability to use personal experience in the context of a understanding of the broader collective experience to work effectively in program activities that depend on it – such as peer based program delivery, health promotion and/or leadership. It less a focus on ‘what is a peer’ and more a focus on ‘what does a peer need to be able to do’. It involves being able to bridge differences, differences between peers and between peers and non-peers in the system
- #12 Most evaluation only looks at two things: quality process in the activities and limited measures of impact, which would fall under the first of four key functions we identified: influence. This is generally as far as the more linear program logics go – and is often simplified to knowledge or behaviour What this misses is the broader community level influences we see - but don’t measure - in our community programs
- #13 To achieve broader community level influence – peer programs depending on the quality and depth of their engagement with those communities – which is our second function. This is why we have a multiplying sign – without strong and sustained engagement – peer programs will have little effectiveness no matter how good they may be. A key reason for this is our next function… .
- #14 Learning and Adaptation We found that peer and community based programs are constantly learning about changes in their target community. This is not just through the running of peer programs, but also peers directly in their own lives, and the organisation as a whole. This is where the learning of peer programs needs to be shared and understood within the organisation so the organisation can support and guide a constantly adapting peer program. And so the organisation as a whole needs to be engaged with that community diversity and dynamism – not just the peer programs. They use insights obtained via practitioner and organisational learning and adapt their approach to maintain or enhance influence.
- #15 But that knowledge and insight is a key asset for the whole sector and policy environment They can also package up that knowledge and use it to influence their partners and stakeholders in the policy system. This is something that is generally missed or ignored in most of the literature on peer based programs, and rarely recognised in the Peer programs do not work in isolation in a broader system – but have a unique insight and perspective into communities as well as interpreting and understanding epidemiological and social research and new technologies and what it may mean for work with their communities.
- #16 The last function is alignment, which refers to the way programs recognise signals from the policy system and adapt their activities to contribute value to the overall prevention system. For example – the more peer insights and leadership is valued, the more influence on the broader policy and sector system, and the more alignment may be achieved for community programs and policy change. It is also about organisations and peer programs seeing changes in the policy and sector – understanding what this will mean for their communities - and identifying what strategies or changes they will need to undertake to maintain their influence in improving the health of their communities. This diagram only shows key connections and flows – and could easily become as detailed as the very complex diagrams we started with across the four programs we were working with. But in essence there seemed to be these four key functions we needed to watch…
- #18 We are now working with the programs to develop and trial indicators related to each of those functions What are the indicators In this framework, indicators are things that might happen or be observed that would increase a program manager or funder’s confidence that the key functions are being fulfilled. - this includes but not limited to direct influence on individuals What we find is that much of these are already present within the scope of the programs, but not recognised as signals for evaluation. So not so much ‘more data’ but nuancing and valuing what we have - and using it much more strategically. How the indicators can be used The indicators act as pegs for collecting and relating together many diverse sources and kinds of knowledge, both qualitative and quantitative, that help answer these key questions: Are programs doing what they’d need to do to improve their influence on communities and policy environments understood as complex systems? Are the programs’ understandings of those complex systems close enough to reality to sustain effective action within them? Can we explain how the program works to external stakeholders and justify their investment of resources (funding, trust, etc) in it? Are program activities and other programs in the sector working effectively together to achieve the goals of state and national strategies on HIV and hepatitis C? They are intended to be used to refine and refocus existing evaluation indicators and activities.
- #19 The indicators are flexible, enabling them to be used in different ways: As objectives in strategic planning for program activities; As topic guides for facilitating team, program and organisational knowledge practices – such as team meetings or planning workshops; In combination with each other to craft internally coherent, politically effective narratives that convey to external stakeholders a sense of the complex relationships that the program is seeking to influence or activate; To plan how different programs or activities could better work together.