Working on Program Implementation and Fidelity: The DEBI Experience
Working on Program Implementation and Fidelity: The DEBI Experience Jonny F. Andia, Ph.D Behavioral Scientist 2011 National HIV Prevention Conference – Atlanta, GA August 14th – 17th 2011 National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Division of HIV/AIDS Prevention
Background• Evidence-based interventions (EBIs) are part of the National HIV/AIDS Strategy to reduce the incidence of HIV in the U.S. and territories.• The CDC/DEBI project disseminates over 23 EBIs with multiple national partners.• Correct adoption of EBI programs, fidelity and quality assurance (QA) are of paramount importance.
Purpose of the Presentation• Describe implementation tools and quality assurance procedures related to EBIs and other Public Health Strategies (PHS)• Describe the EBI dissemination process and methods used by CBA (capacity building assistance) partners to help implement and assess fidelity among CBOs (community-based organizations)
The DEBI Project• This project disseminates evidence-based HIV prevention interventions into prevention practice.• DEBI’s goal is achieved by development and coordination of a national-level strategy providing quality assurance related to: – Training – Technical assistance – Providing national infrastructure and support for CBA activities to state health departments and CBOs
DEBI Model of Dissemination, Implementation and Fidelity* INTERVETNION PROGRAM HEALTH PACKAGE DESIGN MARKTING STRATEGY TRAINING STRATEGIC PLANING CAPACITY BUILDING ASSISTANCE, TA & POLICY AND FIDELITY INCENTIVES EVALUATION & QUALITY CONTROL* Charles Collins, et al., The diffusion of Effective Behavioral Interventions Project: Development, Implementation and Lessons Learned. Moving Science Into Practice, AIDS Education and Prevention (SS) Vol. 18, Spl A. August 2006
The Intervention Package Design• CDC develops and evaluates prototype training materials with members of the target population and in collaboration with original researchers to ensure transfer of science-based technology into practice. – Promotional videos – Instructional videos – Detailed protocols – Evaluation guides, including sample evaluation tools – Technical assistance guides – Sample health education materialsNote: These should be living documents which may be periodically updated so as to improve implementation.
Training Delivery Strategy• A range of pedagogical methods and learning strategies based on adult learning practices are used to transfer science-based information to CBOs and health departments, among them: – Face-to-face training – Virtual classrooms – Web-conference technology – Individual on-line instruction
Training DeliveryInstitutions, organizations and individuals that partner with CDC to disseminate EBIs:• STD/HIV Prevention Training Centers (PTCs)• Capacity Building Assistance (CBA) providers• AIDS Education Training Centers (AETCs)• The original researchers• Danya International, Inc. and or other contractual services
DEBI Training Logistics and Planning• Activities provided by DEBI contractors (i.e., Danya International) to plan, coordinate, support and monitor training services and outcomes of diffusion of EBIs
DEBI Training Results (26 Interventions)Trainings From 1/1/2002 to 5/31/2011 (approximately 8 years) # of trainings # Participants # Employees # Employees CBOs HDs 1142 18,531 4,410 1,027Trainings From 9/1/2010 to 6/30/2011 (8 months) # of trainings # Participants # Employees # Employees CBOs HDs 169 2,540 664 114
Capacity Building and Technical AssistanceCapacity building assistance includes training, TA, and information and technology transferTypes:• CDC project officers• CDC science application team• CBA Request Information System (CRIS) **• Behavioral and Social Science Volunteer Program• Capacity building assistance organizations (CBAs)• PTC trainers may offer phone technical assistance
CRIS – Technical AssistanceThe CRIS system is a web-based system that can be accessed 24 hours, 7 days a week.Key features include:• Monitoring and tracking TA requests• Provides ongoing status reports and a model of communication among the CBA provider, program consultants, project officers and behavioral experts.• Allows requests for additional services from other CBA providers• Assesses the impact and effectiveness of the TA provision
Technical Assistance (CRIS) and Other Related Trainings Events (TEC)Technical Assistance (CRIS) for CBOs (Nov 2009 to Jun 2011) Total TA TA for CBOs TA for EBIs EBI/PHS General requests and PHS implementation evaluation 515 337 (65%) 166 40 44Other related trainings/TEC (Nov 2009 to Jun 2011) Total TEC EBI/PHS Courses on EBI/PHS General trainings support implementation M&E evaluation courses of EBI/PHS 302 74 (15%) 13 1 15
Fidelity Tools• Fidelity tools are usually created by TA experts to assess proper implementation of an intervention in multiple/different conditions• Fidelity tools are an integral part of any needs assessment process (what is needed to maximize efficiency during implementation)• Issues of adaptation are identified early in the implementation process
Fidelity Tools Assessment Areas• Organization (is the agency ready for the intervention?)• Governance (proper organizational support for the intervention)• Programmatic infrastructure (who is doing what in the intervention?)• Workforce and professional development• Resources and support These inputs are ‘required’ to implement an EBI!
Evaluation Field Guides for CBOs & HDs STAGE 1 (MACRO) STAGE 2 (JSI) STAGE 3 (PEB)1. SISTA 13. WILLOW 21. SHIELD2. RAPP 14. START 22. CLEAR3. VOICES 15. CONNECT4. HEALTHY RELATIONSHPS 16. SIHLE5. PROMISE 17. CRCS6. POL 18. MIP (English & Spanish)7. MPOWERMENT 19. PfH8. 3MV 20. PARTNER SERVICES9. FOCUS ON YOUTH10. STREET SMART11. SAFETY COUNTS12. SISTA TO SISTA
Evaluation: Specialized Capacity Building Tools
Results/Conclusions• Capacity building and technical assistance experts (CDC and CBA/PTC providers) develop capacity building tools and other fidelity tools to assess the proper implementation of interventions• Provision of CBA during dissemination and implementation of EBIs is fundamental to ensure optimal levels of fidelity and outcomes
Co-authors• Juli Powers, MPH. JSI Research and Training Institute, Inc.• Ricky Tavares, MPH. National Community Health Partners• Naima Cozier, MSPH. JSI Research and Training Institute, Inc.• Charles Collins, Ph.D. NCHHSTP/CBB
Acknowledgements• Rashad Burgess, MPH. CDC/CBB• Aisha Gilliam, Ed.D. CDC/CBB• Ashley Gamayo, MPH. CDC/CBB (ASPH Fellow)• JSI Research & Training Institute, Inc.• National Community Health Partners (NCHP)
Thank you!For more information please contact Centers for Disease Control and Prevention1600 Clifton Road NE, Atlanta, GA 30333Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: firstname.lastname@example.org Web: http://www.cdc.govThe findings and conclusions in this report are those of the authors and do not necessarily represent the official position ofthe Centers for Disease Control and Prevention. National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Division of HIV/AIDS Prevention