More Related Content
What's hot
What's hot (19)
Similar to Preventing failed interventions
Similar to Preventing failed interventions (20)
Recently uploaded
Recently uploaded (20)
Preventing failed interventions
- 1. Preventing Failed Interventions We got into our “we know best mode” Adriaan van 't Spijker Orla Mooney
- 2. #GE16 PhD PhD Project: Designing a Self-Management (SM) Intervention for Teens with Epilepsy Phase 1: Systematic Review = mixed support for interventions What factors might contribute to failed interventions?
- 3. #GE16 Skills Assessment To build on my skills from HNC project
- 4. #GE16 Background of Training 40 hours of class attendance Group sessions Group presentation Certification: ‘Preventing Failed Interventions in Behavioural Research’ Erasmus MC, Rotterdam 19th – 23rd Feb 2018 Prerequisites: Basic knowledge of statistics & of study design Participant Profile: Researchers of psychological Interventions
- 6. #GE16 Preventing Failed Interventions Elements This course elaborates on intervention development, implementation and evaluation in the field of medical psychology. The focus is on learning from encountered difficulties, mistakes and failures from previous research and researchers. Topics Example of failed intervention Sample size calculations Methods of administration Increasing Response rates Intervention Mapping SPIRIT 2013 checklist/CONSORT 2010 checklist Biases in outcome measures Monitoring Interventions
- 7. #GE16 Examples of my Main Learning • Examples of failure reasons • Intervention Mapping
- 8. #GE16 • Examples of Failure Reasons • Some examples of failure reasons • Buy-in from externals (e.g. consultants, nurses) • Personnel changes • Changes in treatment during study • Incentives for participants • Sample size (e.g. Hepatitis study) • Managing drop-outs • Relevant inclusion/exclusion criteria • Managing multi-centre interventions • Monitoring and evaluating the trainer
- 9. #GE16 • Examples of Failure Reasons • Some examples of failure reasons (Kok, 2014) cont… • Incorrect identification of the change objectives • Inappropriate choice of methods or applications • Inadequate implementation in terms of completeness and fidelity of the programme being delivered NB: All 3 are often the result of insufficient participation of the target population, implementers, decision makers etc…
- 10. #GE16 Preventing Failed Interventions I N T E R V E N T I O N M A P P I N G (Bartholomew et al., 2011) P L A N N I N G R E S E A R C H T H E O R Y
- 11. #GE16 What have I learnt • The importance of the needs of the target population • The importance of intervention mapping • The importance of both monitoring and evaluating • The importance of validated checklists and tools • To always be mindful of biases • We need to know about the studies that fail so that we can learn from them • How useful my student internships have been
- 12. #GE16 What’s Next • To use intervention mapping to develop phase 2 – 4 of my project • I would be interested in a course focusing more in-depth on mapping • Developing my qualitative skills with the DCU Summer School • Developing my skills around behaviour change • Learning about self-regulation and it’s role in SM
- 13. Final Comment Behaviour change is difficult, if it was easy it would have already happened… The highest chance for success can be expected from a theory and evidenced based process. Intervention Mapping is one such protocol to support success (Kok, 2014)
Editor's Notes
- Good afternoon, I am Orla Mooney and welcome to my presentation on developing my skill for intervention design
- The overall aim of my PhD (subject to change) is to develop a self-management intervention for teenagers living with Epilepsy. When working on my systematic review, other reviews in this area for children and young people have identified mixed support for the small number of interventions that have been developed and this is why I wanted to develop my skills in the area of intervention design
- This is a diagram of my skills assessment and from my work on a student internship across 3 summers I noticed gaps in interventions being underpinned by theory, a lack of tools online for intervention design and seemingly a mismatch between what participants want and need from interventions (that relates back to the comment on slide one, where an intervention designer explained sometimes we go into we know best mode. So I really wanted to hon my skills and learn more about intervention design as understanding possible reasons of failure, can help me from phase 2 not just at the end at the design phase.
- To develop this skill of intervention design, I took a 5 day course, ‘Preventing Failed Interventions in Behavioural Research’ in Erasmus MC in Feb which involved 40 hours of class attendance, in class group sessions and a group presentation on the final day
- My expectations were that I would be informed of best practice in intervention design and implement what I learnt into the next phases of my research and also for research I may do, after my PhD
- This course elaborates on intervention development, implementation and evaluation in the field of medical psychology. The focus is on learning from encountered difficulties, mistakes and failures from previous research and researchers. Some of the topics addressed were: exampled of failed interventions that the trainers were involve in, why we need to do sample size calculations for example, for grants and for power, however we often do it the other way around based on availability of participants, money, time restraints and personnel available. We looked at the methods of administration in relation to questionnaires (for baseline and evaluations) and how to improve response, for example, incentives, reminders, involving clinicians and also discussed time spent on missing data and drop outs. The intervention mapping I will discuss in more detail next. Monitoring interventions was about the importance of the provider/therapist and how we assume all providers provide the same intervention. Benchmarking therapists was also important and an interesting idea of recruiting less experienced ones, as they are more likely to follow your exact instructions and not feel that they know better and make their own adjustments and monitoring such as some type of recordings of sessions. For biases we explored topics such as social desirability, framing effects and issues around likert scales such as how many labels and should you have a middle label or reverse scales. We also looked at the use of SPIRIT and CONSORT checklists which help you check you have all the relevant information to report on interventions. Finally we had 24 hours to work within groups to develop an intervention protocol and present it to the class.
- For example, an intervention for Hep C was to run across 14 centres, it required 300 participants, 150 in each arm and ended up with 33 recruits and only 5 completed. Some of the reasons they identified: changes of personnel, buy in from consultants however it was nurses who would be involved, there were only 400 eligible participants to recruit from their inclusion/exclusion criteria, they initially excluded people with HIV but had to remove that, most participants were ex-drug addicts who can have issues with adherence
- For example, an intervention for Hep C was to run across 14 centres, it required 300 participants, 150 in each arm and ended up with 33 recruits and only 5 completed. Some of the reasons they identified: changes of personnel, buy in from consultants however it was nurses who would be involved, there were only 400 eligible participants to recruit from their inclusion/exclusion criteria, they initially excluded people with HIV but had to remove that, most participants were ex-drug addicts who can have issues with adherence
- Intervention mapping is a protocol for developing theory-based and evidence-based health promotion programs. Intervention Mapping describes the process of health promotion program planning in six steps: the needs assessment based on the PRECEDE-PROCEED model. I found this an excellent tool to learn about and essential for the next phases of my project. It provides 6 clear steps to follow, for example, beginning with needs assessments which I will implement with a qualitative phase where I will involve stakeholders (through interview or online focus groups) to identify what self-management they currently do and what their self-management needs are – what they feel is not being met (not what I think they are). From this I can then identify the relevant behaviour and environmental change, performance objectives and change determinants. With this in place, I can identify which change theory will best underpin the intervention. For the design of the intervention I have learnt from the course that it is not just enough to involve participants only at the qualitative stage and that it is important that they get to have an input in the design also, and any persons who will be involved in the delivery. This will further map the actual needs with the most suitable self-management tools/modules for the program. Moving on to implementation and evaluation. The evaluation is an important step as from reading much literature it is an area that is not reported regularly on and can contribute to the mixed results.