Your SlideShare is downloading. ×
A draft report of the Evidence Synthesis and Application for Policy and Practice project
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

A draft report of the Evidence Synthesis and Application for Policy and Practice project

382
views

Published on

Graham Brown (Australian Research Centre in Sex, Health and Society) discusses the importance of maintaining a strong evidence base for health promotion.

Graham Brown (Australian Research Centre in Sex, Health and Society) discusses the importance of maintaining a strong evidence base for health promotion.

Published in: Health & Medicine

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
382
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ESAPP ReviewA draft report of theEvidence Synthesis and Application for Policy and Practice projectMay 2013Graham Brown, Kylie Johnston and Jeanne EllardAustralian Research Centre in Sex, Health and Societywww.latrobe.edu.au/arcshsMelbourne, Australia
  • 2. HIV response in Australia• Australian HIV response has long recognised that interventions working acrossmultiple social, political, economic, behavioural and health serviceconditions, operating within supportive environments, are more likely to affectbehaviour than those interventions working at one level• Resurging and emerging epidemics• Role of antiretroviral (ARV) treatments in preventing HIV transmission• Continued barriers at a structural level reducing or undermining impact• Recognising the need to better integrate biomedical, individual, community andstructural approaches for HIV prevention - coined “combination prevention”
  • 3. Evidence gaps• Shared evidence base is not consistent across strategies or has not beenmaintained as the contexts have continued to change.• Understanding of what works - but less so why, or in what combination.• This undermines the strength of programs and organisations, and the capacity toadapt to changing environments with confidence.• Highlighted within Implementing the UN Declaration Report and Melbourne Declaration
  • 4. identify the areas of HIV prevention where the published evidence ofeffectiveness and quality practice is most, modest, or least developed; Section 2.0 of full reportidentify the monitoring and evaluation methods used in day to day practicein community organisations to contribute to that evidence (Australia andsimilar epidemics); Section 3.0 of full reportreview of capacity-building approaches in Australia and internationally toincrease the quality of evidence being developed in community-based HIVhealth promotion; Section 4.0 of full reportdevelop a draft Monitoring, Evaluation and Learning framework forcommunity HIV prevention to support building evidence for policy andpracticeSection 5.0 of full reportDevelop an draft example application of the Monitoring, Evaluation andLearning Framework to community based HIV prevention and healthpromotionSection 6.0 of full report
  • 5. Evidence most, modest and least developed• Review of systematic reviews, economic reviews, narrative reviews andcommentaries on the evidence to guide the prevention of sexualtransmission of HIV in concentrated epidemics (2005+).• Published evidence from research and practice (reduced to ~130 articles)• Additional focus - three priority groups identified due to the likely impact oftesting and treating approaches as well as experiencing resurging or emergingepidemics.– PLHIV -Gay Men -Priority culturally and linguistically diverse (CALD) communities
  • 6. Generally Implementation evaluation and quality practice indicators withspecific target groupsExample program Evidence on what doesor does not workEvidence on how itworks (including how toadapt to context)Gay men PLHIV Priority CALDcommunities inwestern countriesHealth promotionSystemshow the interventionsinteract and impacttogetherLeast developed Least developed Least developed Least developed Least developedStructural Reduction of HIVstigma, policy reformLeast developed Least developed Least developed Least developed Least developedCommunity Mass media, socialmedia, communitymobilisationModeratelydevelopedLeast developed –varies across modes andtarget groupsModeratelydevelopedLeast developed Least developedSmall Group Structured peer basedworkshopsMost developed moderate– varies acrossmodes and target groupsModeratelydevelopedModeratelydevelopedLeast developedIndividual Peer and professionalcounsellingMost developed Most developed Most developed Most developed ModeratelydevelopedBiomedicalPreventionIncreased testing,Treatment asPreventionMost developed ModeratelydevelopedLeast developed Least developed Least developedSummary of where published evidence about HIV prevention and health promotion is most, moderately or least developed
  • 7. Recommendations for Improving Evidence BaseTwo key interrelated factors:• Research: Intervention research trials that use a broad range of rigorous designs appliedappropriately to interventions at different levels of health promotion, and investigate what works aswell as why it works and in what context.• Practice: Stronger implementation research within CBOs with a focus on program theory, qualitypractice indicators, and development of sustainable evaluation and quality improvementapproaches that recognise the need to continuously adapt and reorient programs.Without these reorientations in both research and practice, evidence will =• Limited to the impact of parallel but unconnected strategies• Provide little insight to what are the most effective leverage points, and what to change as the situationevolves.
  • 8. Where MEL&QI is most, modest and least developed• Rapid review of current practices used in Australia (building on previous work undertakenby AFAO in 2008)• Rapid review of evaluation practice in international contexts similar to Australia(primarily Europe and North America).• Reviewed the published work, abstracts of key conferences attended by HIV educators inAustralia and internationally and supplement this with other targeted online searcheswith organisations. (~reduced to ~100 documents)• While not a complete audit of all work undertaken - reasonable overview of most keydevelopments in monitoring and evaluation since 2008 with least intrusion on thecommunity sector organisations
  • 9. Intervention level Strategies (examples) Process and qualitypractice indicatorsImpact Indicators Combinationprevention or systemwide synergy indicatorsStructural Policy and law reform,advisory structures,Moderately developed Least developed Least developedCommunity Community engagementand mobilisationModerately developed Least developed Least developedOnline Social Media Least developed Least developed Least developedMass media Moderately developed Moderately developed Least developedSmall Group Structured peer basedworkshopsModerately developed Moderately developed Least developedIndividual Peer Counselling Models Most developed Moderately developed Moderately developedProfessional CounsellingmodelsMost developed Most developed Moderately developedSummary of where MEL approaches are most, moderately or least developed
  • 10. CBO Capacity Building Initiatives• In broad terms, most programs aimed to move organisations or sectors throughstages of evaluation capacity– compliance (fulfilling funding source requirements),– investment (beyond compliance, evaluation is used to improve programs and issupported by leadership), and– advancement (beyond investment, evaluations are increasingly ambitious and contributeto prevention theory and practice). (Gilliam et al., 2003)• Full Report gives examples of Australian and International Initiatives– (incl Acon PEKM)
  • 11. CBO Capacity Building InitiativesThese and other similar initiatives have generally included among their aimsto increase:• capacity to determine why an intervention works, not just if it works,• capacity for continuous quality improvement approaches, and• understanding of, and methods to, identify impact within a combinationprevention or health promotion system,• documenting and sharing of the knowledge and learning.
  • 12. Monitoring, Evaluation and Learning (MEL) and QualityImprovement (QI) frameworkThe framework endeavours to acknowledge:• the complexity of the evolving health, social and political systems in whichHIV prevention operates;• the strengths of the partnership response and combination approaches;and• the rigour of program logic, program theory, quality improvement andsystems thinking.
  • 13. Priority Community YIndividualGroupCommunityStructuralIndividual andclinical focusservicesTargeted Communitydevelopment and socialinfluencePeer group developmentand network focusedprojectsCommunity TargetedSocial marketingCommunity venues andsettings based engagementOrganisational andsystemic changeStructural , policy andsocial changeMass Media Social marketingPopulationHealthOutcomeReducedtransmissionandimpactofHIVIntegrated Combination of Health Promotion Actions and Outputs
  • 14. Priority Community YIndividualGroupCommunityStructuralIndividual andclinical focusservicesTargeted Communitydevelopment and socialinfluencePeer group developmentand network focusedprojectsCommunity TargetedSocial marketingCommunity venues andsettings based engagementOrganisational andsystemic changeStructural , policy andsocial changeMass Media Social marketingPopulationHealthOutcomeReducedtransmissionandimpactofHIVIntegrated Combination of Health Promotion Actions and OutputsAcross Priority Communities
  • 15. Integrated Combination of Health Promotion Actions and OutputsPriority Community YIndividual andclinical focusservicesTargeted Communitydevelopment and socialinfluencePeer group developmentand network focusedprojectsCommunity TargetedSocial marketingCommunity venues andsettings based engagementOrganisational andsystemic changeStructural , policy andsocial changeMass Media Social marketingIndividualGroupCommunityStructuralPopulationHealthOutcomeReducedtransmissionandimpactofHIV
  • 16. Priority Community YIndividual andclinical focusservicesTargeted Communitydevelopment and socialinfluencePeer group developmentand network focusedprojectsCommunity TargetedSocial marketingCommunity venues andsettings based engagementOrganisational andsystemic changeStructural , policy andsocial changeMass Media Social marketingSector WideNational StrategyoutcomesLonger term /Combined ProgramLevel ImpactShort Term/Individual ProjectLevel ImpactPopulationHealthOutcomeIndividualGroupCommunityStructuralImproved relevantknowledge, attitudes, skills, and selfefficacyEnhanced qualitypractice indicatorsIndicators ofstrengthenedcommunitycapacity andresponsesProject levelquality andimpactindicatorsIncreased access tohealthservices, testing andtreatmentImpact on peernorms andexperienceParticipation ofaffectedcommunitiesIncreased healthpromoting socialnorms within prioritycommunitiesIncrease in levels ofprotective sexual riskbehaviour andtesting in at riskgroupsStrengthenedintegration acrosshealth promotionstrategiesProgram levelquality andimpact indicatorsIncreased indicators ofsustained communityresponses amongpriority populationsIncreased sustainedtesting and treatmentuptakeReducedtransmissionandimpactofHIVReduced riskbehavioursDecreaseundiagnosed HIVPreventionsystem leveloutcomes (linkedto National HIVStrategy and Targets)Improve QoL ofPLWHIVIncrease PLWHIV onTreatment with UVLStrengthened systemsin research, evaluationand workforceReduced incidence ofHIVIntegrated Combination of Health Promotion Actions and OutputsAcross Priority Communities
  • 17. Priority Community X Priority Community Y Priority Community ZIndividualGroupCommunityStructuralSector WideNational StrategyoutcomesLonger term /Combined ProgramLevel ImpactShort Term/Individual ProjectLevel ImpactPopulationHealthOutcomeOrganisational andsystemic changeTargeted Communitydevelopment and socialinfluenceIndividualand clinicalfocus servicesPeer group developmentand network focusedprojectsCommunity TargetedSocial marketingIndividual andclinical focusservicesStructural , policy andsocial changeMass Media Social marketingCommunity venues andsettings based engagementImproved relevantknowledge,attitudes, skills,and self efficacyEnhanced qualitypractice indicatorsIndicators ofstrengthenedcommunitycapacity andresponsesProject levelquality andimpactindicatorsIncreased access tohealthservices, testing andtreatmentImpact on peernorms andexperienceParticipation ofaffectedcommunitiesIncreased healthpromoting socialnorms within prioritycommunitiesIncrease in levels ofprotective sexual riskbehaviour andtesting in at riskgroupsStrengthenedintegration acrosshealth promotionstrategiesProgram levelquality andimpact indicatorsIncreased indicators ofsustained communityresponses amongpriority populationsIncreased sustainedtesting and treatmentuptakeReducedtransmissionandimpactofHIVReduced riskbehavioursDecreaseundiagnosed HIVPreventionsystem leveloutcomes (linkedto National HIVStrategy and Targets)Improve QoL ofPLWHIVIncrease PLWHIV onTreatment with UVLStrengthened systemsin research, evaluationand workforceReduced incidence ofHIVIntegrated Combination of Health Promotion Actions and OutputsAcross Priority Communities
  • 18. Priority Community X Priority Community Y Priority Community ZIndividualGroupCommunityStructuralSector WideNational StrategyoutcomesLonger term /Combined ProgramLevel ImpactShort Term/Individual ProjectLevel ImpactPopulationHealthOutcome..Organisational andsystemic changeTargeted Communitydevelopment and socialinfluenceIndividualand clinicalfocus servicesPeer group developmentand network focusedprojectsCommunity TargetedSocial marketingIndividual andclinical focusservicesIndividualand clinicalfocus servicesStructural , policy andsocial changeMass Media Social marketingCommunity venues andsettings based engagementImproved relevantknowledge,attitudes, skills,and self efficacyEnhanced qualitypractice indicatorsIndicators ofstrengthenedcommunitycapacity andresponsesProject levelquality andimpactindicatorsIncreased access tohealthservices, testing andtreatmentImpact on peernorms andexperienceParticipation ofaffectedcommunitiesIncreased healthpromoting socialnorms within prioritycommunitiesIncrease in levels ofprotective sexual riskbehaviour andtesting in at riskgroupsStrengthenedintegration acrosshealth promotionstrategiesProgram levelquality andimpact indicatorsIncreased indicators ofsustained communityresponses amongpriority populationsIncreased sustainedtesting and treatmentuptakeReducedtransmissionandimpactofHIVReduced riskbehavioursDecreaseundiagnosed HIVPreventionsystem leveloutcomes (linkedto National HIVStrategy and Targets)Improve QoL ofPLWHIVIncrease PLWHIV onTreatment with UVLStrengthened systemsin research, evaluationand workforceReduced incidence ofHIVIntegrated Combination of Health Promotion Actions and OutputsAcross Priority Communities
  • 19. Priority Community X Priority Community Y Priority Community ZIndividualGroupCommunityStructuralSector WideNational StrategyoutcomesLonger term /Combined ProgramLevel ImpactShort Term/Individual ProjectLevel ImpactPopulationHealthOutcome...Organisational andsystemic changeTargeted Communitydevelopment and socialinfluenceIndividualand clinicalfocus servicesPeer group developmentand network focusedprojectsCommunity TargetedSocial marketingIndividual andclinical focusservicesIndividualand clinicalfocus servicesStructural , policy andsocial changeMass Media Social marketingCommunity venues andsettings based engagementImproved relevantknowledge, attitudes, skills, and selfefficacyEnhanced qualitypractice indicatorsIndicators ofstrengthenedcommunitycapacity andresponsesProject levelquality andimpactindicatorsIncreased access tohealth services,testing andtreatmentImpact on peernorms andexperienceParticipation ofaffectedcommunitiesIncreased healthpromoting socialnorms within prioritycommunitiesIncrease in levels ofprotective sexual riskbehaviour andtesting in at riskgroupsStrengthenedintegration acrosshealth promotionstrategiesProgram levelquality andimpact indicatorsIncreased indicators ofsustained communityresponses amongpriority populationsIncreased sustainedtesting and treatmentuptakeReducedtransmissionandimpactofHIVReduced riskbehavioursDecreaseundiagnosed HIVPreventionsystem leveloutcomes (linkedto National HIVStrategy and Targets)Improve QoL ofPLWHIVIncrease PLWHIV onTreatment with UVLStrengthened systemsin research, evaluationand workforceReduced incidence ofHIVIntegrated Combination of Health Promotion Actions and OutputsAcross Priority CommunitiesInputs/ ResourcesCommunityOrganisationsand advocacyClinical andprimary careServices(medical andCounselling)Advisorystructures, Policy, andresourceallocation*ExternalInfluencesSocialDeterminantsSocial DriversCommunitycapacity ,strength andparticipationBiomedicaltesting,treatment andpreventiondevelopmentsPopulationimpacts oftesting andtreatmentsSocial capitalStigma anddiscriminationPartnership*,GovernanceandLeadership*GuidingPrinciplesand ethicsHuman rightsResearchorganisations
  • 20. Inputs/ ResourcesCommunityOrganisationsand advocacyClinical andprimary careServices(medical andCounselling)Advisorystructures,Policy, andresourceallocation*ExternalInfluencesSocialDeterminantsSocial DriversCommunitycapacity ,strength andparticipationBiomedicaltesting, treatment andpreventiondevelopmentsPopulationimpacts oftesting andtreatmentsSocial capitalStigma anddiscriminationIndividual andinterpersonaltheoriesStructural andSystem theorySocial /BehaviouraltheoriesSocial andEpidemiologicalResearchProject, Program andsystem level evidenceand evaluationContinuous Quality Improvement, refinement of practiceguidelines and standards, and development of workforce*Partnership*,GovernanceandLeadership*Information Systems* (Monitoring,Evaluation and Learning)Priority Community X Priority Community Y Priority Community ZIndividualGroupCommunityStructural*Prevention SystemStrengthening buildingblocks identified by WHOSector WideNational StrategyoutcomesLonger term /Combined ProgramLevel ImpactShort Term/Individual ProjectLevel ImpactPopulationHealthOutcome...Organisational andsystemic changeTargeted Communitydevelopment and socialinfluenceIndividualand clinicalfocus servicesPeer group developmentand network focusedprojectsCommunity TargetedSocial marketingIndividual andclinical focusservicesIndividualand clinicalfocus servicesStructural , policy andsocial changeMass Media Social marketingCommunity venues andsettings based engagementImproved relevantknowledge, attitudes, skills, and selfefficacyEnhanced qualitypractice indicatorsIndicators ofstrengthenedcommunitycapacity andresponsesProject levelquality andimpactindicatorsIncreased access tohealthservices, testing andtreatmentImpact on peernorms andexperienceParticipation ofaffectedcommunitiesIncreased healthpromoting socialnorms within prioritycommunitiesIncrease in levels ofprotective sexual riskbehaviour andtesting in at riskgroupsStrengthenedintegration acrosshealth promotionstrategiesProgram levelquality andimpact indicatorsIncreased indicators ofsustained communityresponses amongpriority populationsIncreased sustainedtesting and treatmentuptakeReducedtransmissionandimpactofHIVReduced riskbehavioursDecreaseundiagnosed HIVPreventionsystem leveloutcomes (linkedto National HIVStrategy and Targets)Improve QoL ofPLWHIVIncrease PLWHIV onTreatment with UVLStrengthened systemsin research, evaluationand workforceReduced incidence ofHIVIntegrated Combination of Health Promotion Actions and OutputsAcross Priority CommunitiesInternationalpractice anddevelopmentsGuidingPrinciplesand ethicsHuman rightsResearchorganisations
  • 21. Exampleinputs andresourcesExampleProjectExample indicators for Project level MEL&QI(preferably drawn from project’s own project logic)Example Program Level MEL&QI(such as range of peer based projects)Example qualitypracticeindicatorsExampleIndicators ofInter-projectquality linksExampleProjectOutputIndicatorsExample Project level impactindicators.(immediate to 3 month)Example Programlevel qualityindicatorsExample Inter-program quality linksProgram levelimpact indicators(3 to 12 month)CommunityorganisationresourcesPrinciples ofpeer basedprogramsPeer basedstaff andvolunteersEvaluationfrompreviousprogramsSmall GroupLevel Project:eg- PeerGroupworkshop forgay menQuality practiceinvolvement of targetgroup in development andimprovement.Satisfaction measuresGroup interaction anddynamics indicatorsEvidence of reciprocallearning betweenparticipantsProportion of participantswho complete workshopsPeer referrals /recommendationsReferrals fromoutreach, onlineinitiatives,counsellingDiscussion oruse of socialmarketingcampaign withinworkshopCommunityvolunteerengagementindicatorsNumber ofworkshopsconductedAverage numberof participantscompletingworkshopsAlignment ofintended targetgroup andactivityparticipantsA workshop would be focused on onlythree or four of a set of project level impactindicators – depending on the focus of theworkshop. The following is an example of aset of indicators from which a workshopmay draw:1. Increase in sexuality related health literacy and supportseeking knowledge.2. Increase in knowledge and confidence to interact indiverse and sexualised environments (eg online, SOPV,etc).3. Increase in skills and confidence to negotiate sexualinteractions including safe sex practices4. Increase in confidence to manage HIV disclosure insexual and social settings5. Increase in knowledge and confidence regarding sexualtechnique and repertoire6. Increase in confidence to develop relationships(intimate and friendship).7. Increase in indicators of participants influencing theirpeers regarding peer program messagesIndicators of participantsinfluencing their peers inrelation to programaimsIncreased indicators ofsustained communityresponses amongpriority populationsIndicators of communitylevel engagement withstrategiesVolunteer recruitmentfrom peer programsStrategic links betweenpeer group project andcommunitydevelopment projectsStrengthenedintegration andstrategic links acrosspeer based programsand other promotionstrategiesReferrals to and fromvenue outreach, onlineinitiatives, orcounsellingIncreased healthpromoting social normswithin prioritycommunitiesIndicators of testing andtreatment uptakeIncrease in levels ofprotective sexual riskbehaviour and testingamong programparticipantsApplication of MEL&QI framework to a hypothetical peer group workshop for gay men
  • 22. Project level quality,monitoring andevaluationProgram Level quality,monitoring andevaluationPrevention system levelquality, monitoring,surveillance and evaluationProject /Service staff Yes Possibly NoAgency/Program Yes Yes PossiblyExternal evaluators Possibly Yes PossiblyHealth Services Data Possibly Yes YesEpidemiology and SocialResearch Centres /DepartmentNo Possibly YesGuidelines for responsibility for collecting and summarising MEL data
  • 23. Final CommentsThis is a draft and at a conceptual level– Draft summary and full report available for comment– Presented as a discussion monograph in JulyPossibly more than ever our community sector needs to• look at frameworks and approaches to building and expanding the evidencebase, particularly where it is less developed• Recognise that the projects and programs will continuously evolve and change• Understanding the what, why, and in what combination or system of approaches• Turning this into a useable shared evidence base

×