Mind the Gap:Risk of Bias in Pediatric TrialsStaR Child Health SummitWinnipeg, ManitobaMay 7, 2012Michele Hamm, MScAlberta...
Conflict of Interest Disclosure                          Michele Hamm             Mind the Gap: Risk of Bias in Pediatric ...
Introduction There is a growing body of literature documenting  the limitations and methodological flaws of  pediatric re...
Risk of Bias Risk of bias relates to internal validity Design features associated with effect estimate  magnitude:     ...
Progress of the RoB SDG2nd StaR Child Health Summit: Empirical    evidence showing that pediatric RCTs are at    high/uncl...
Mixed Methods Study ofPediatric Trialists Objective: To determine the barriers and  facilitators faced by pediatric trial...
Survey Methods Internet-based survey (SurveyMonkey/REDCap) Surveyed corresponding authors of pediatric trials  published...
Survey Results 23.0% response rate (186/807) 44.6% trained in medicine; 35.5% had formal  research training Median numb...
Survey ResultsBarriers: Lack of sufficient funding (70.3%); Overwhelming volume of literature (63.1%); Logistics make i...
Interview Methods Semi-structured interviews building upon  quantitative survey data MICYRN survey respondents invited t...
Interview Results 13 interviews conducted Necessary to augment original sample:    3 respondents from survey, 3 from MI...
Interview ResultsIndividual Factors - Barriers Knowledge/training regarding research methods   Lack of formal training –...
Interview ResultsIndividual Factors – Barriers“Probably we don’t look at, we don’t know all the bias that  can happen in a...
Interview ResultsIndividual Factors – Facilitators Sense of ownership    Opportunity to generate enthusiasm, gain suppor...
Interview ResultsIndividual Factors – Facilitators“So you really have to take the time to engage people and be the  one th...
Interview ResultsInstitutional Factors – Barriers Negative research culture Lack of recognition of distinction between c...
Interview ResultsInstitutional Factors – Barriers“I think that other people view [research] as kind of a thorn in their   ...
Interview ResultsInstitutional Factors – Facilitators Cohesive study teams    Clinicians, methodologists, research staff...
Interview ResultsInstitutional Factors – Facilitators“We have the help of the research institute and you can have a  perso...
Interview ResultsPolicy Factors – Barriers Funding Ethics review process
Interview ResultsPolicy Factors – Barriers“We all tend to want to make the budget as small as we can to increase  our chan...
Interview ResultsPolicy Factors – Facilitators None
Discussion Internal validity is not a primary concern –  overshadowed by pragmatics and generalizability    Internal val...
Future Directions Objectives: To design and evaluate a tailored KT  intervention to improve methodological rigor in  chil...
Future Directions Consideration of traditional KT interventions    Adaptation for changing researcher behaviour, rather ...
Future Directions
Future Directions
Future Directions
Future Directions
Future Directions
Future Directions
Future DirectionsPilot testing: Trialists, systematic reviewers, trainees Preliminary feedback: content, format, usabili...
Future Directions RoB wiki to be used as a model for other StaR  Child Health SDGs Evaluation of effectiveness    RoB i...
AcknowledgementsSupervisory Committee:   RoB SDG Members:Lisa Hartling            Jamie BrehautTerry Klassen            An...
Upcoming SlideShare
Loading in …5
×

Risk of Bias_StaR Child Health Summit_07May12

2,191 views

Published on

Published in: Health & Medicine, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,191
On SlideShare
0
From Embeds
0
Number of Embeds
1,737
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Risk of Bias_StaR Child Health Summit_07May12

  1. 1. Mind the Gap:Risk of Bias in Pediatric TrialsStaR Child Health SummitWinnipeg, ManitobaMay 7, 2012Michele Hamm, MScAlberta Research Centre for Health Evidence
  2. 2. Conflict of Interest Disclosure Michele Hamm Mind the Gap: Risk of Bias in Pediatric Trials• Received grant/research support from: KT Canada
  3. 3. Introduction There is a growing body of literature documenting the limitations and methodological flaws of pediatric research, specifically risk of bias. The introduction of bias into a trial can lead to the overestimation of treatment benefits or underestimation of treatment harms.
  4. 4. Risk of Bias Risk of bias relates to internal validity Design features associated with effect estimate magnitude:  Sequence generation  Allocation concealment  Blinding - participants/personnel, outcome assessors  Incomplete outcome data  Selective outcome reporting  “Other” sources of bias
  5. 5. Progress of the RoB SDG2nd StaR Child Health Summit: Empirical evidence showing that pediatric RCTs are at high/unclear risk of bias.3rd StaR Child Health Summit:1) Mixed methods study: Why are biased trials being conducted?2) Knowledge translation: Development and pilot testing of a wiki-based educational resource
  6. 6. Mixed Methods Study ofPediatric Trialists Objective: To determine the barriers and facilitators faced by pediatric trialists in the design and conduct of methodologically rigorous trials, using both quantitative and qualitative data. Explanatory mixed methods design  Survey  Semi-structured interviews
  7. 7. Survey Methods Internet-based survey (SurveyMonkey/REDCap) Surveyed corresponding authors of pediatric trials published in 2008 and 2009  Entire sample of Canadian researchers (n=90)  Random sample of international researchers (n=600) Surveyed from MICYRN membership (n=163) Questions to determine: 1) knowledge and awareness of bias 2) perceived barriers and facilitators in conducting trials 3) utility of potential KT strategies for future interventions
  8. 8. Survey Results 23.0% response rate (186/807) 44.6% trained in medicine; 35.5% had formal research training Median number of trials involved in:  As PI: 3 (IQR 1-5)  As part of study team: 5 (IQR 2-10) Most common subspecialties represented:  Public health (8.6%)  Developmental, psychosocial, and learning problems (7.5%)  Mental health or psychiatry (7.0%)
  9. 9. Survey ResultsBarriers: Lack of sufficient funding (70.3%); Overwhelming volume of literature (63.1%); Logistics make it difficult to minimize bias (52.9%)Facilitators: Interest in staying current with literature (93.0%); Opportunities to discuss methods with knowledgeable colleagues (92.8%); Rigorous methods encouraged by colleagues (80.4%)
  10. 10. Interview Methods Semi-structured interviews building upon quantitative survey data MICYRN survey respondents invited to participate in an interview Target sample size of 12 pediatric trialists Questions to determine:  Relationships between participants’ beliefs, behaviours, and attitudes about conducting research on children and appropriate design and conduct of methodologically sound trials
  11. 11. Interview Results 13 interviews conducted Necessary to augment original sample:  3 respondents from survey, 3 from MICYRN, 7 referrals 7 trained in medicine, 4 trained in both medicine and research, and 2 trained in research Subspecialties represented: anesthesiology, clinical epidemiology, critical care, emergency medicine, infectious disease, neonatology, neurology, oncology, psych ology, rheumatology
  12. 12. Interview ResultsIndividual Factors - Barriers Knowledge/training regarding research methods  Lack of formal training – on the job learning  Emphasize certain aspects, overlook others
  13. 13. Interview ResultsIndividual Factors – Barriers“Probably we don’t look at, we don’t know all the bias that can happen in a trial because we don’t check, we don’t believe there’s bias. We may miss some, we may forget some, and then do not report the bias because we don’t know it exists.”“Because there’s almost zero research training in the clinical curriculum for most clinicians these days. Like there’s almost nothing in the med school program, there’s almost nothing in the rehab program – there really needs to be somebody on the protocol who’s got a little bit more training.”
  14. 14. Interview ResultsIndividual Factors – Facilitators Sense of ownership  Opportunity to generate enthusiasm, gain support, and educate colleagues
  15. 15. Interview ResultsIndividual Factors – Facilitators“So you really have to take the time to engage people and be the one that’s proactive, engaging them. Because they’re busy, they might not even know what your study is unless you’re the change agent that really goes out there and talks to them about it and gets them motivated about why you think it’s important.”“Listen to what [your colleagues] need to execute the study so that when you develop your protocol, you’ve built that into the approach. Or, if you couldn’t, you’ve at least had that dialogue with them about how scientifically you can’t be as flexible as might be ideal… so that they at least understand the rationale.”
  16. 16. Interview ResultsInstitutional Factors – Barriers Negative research culture Lack of recognition of distinction between clinical care and clinical research Variable across institutions
  17. 17. Interview ResultsInstitutional Factors – Barriers“I think that other people view [research] as kind of a thorn in their side. It’s something they play along with if they have to and the division head tells them they have to.”“I think a lot of investigators really have a hard time separating what decision they would make clinically from what decision they would make as part of a trial… because the feeling is I want to be convenient to the family, and I really know this stuff because I’m an expert in this clinical area, and I don’t think they realize that there’s a pretty clear demarcation between what you do in clinical care and what you do as research.”
  18. 18. Interview ResultsInstitutional Factors – Facilitators Cohesive study teams  Clinicians, methodologists, research staff  Networks within subspecialties Reliable internal review processes
  19. 19. Interview ResultsInstitutional Factors – Facilitators“We have the help of the research institute and you can have a person for any kind of question or any kind of design that can help, and we have access to those kinds of resources.”“I think the fact that [research network] is there enables you to think of multi-centre RCTs, whereas if it wasn’t there, you’d kind of have to go and find things from scratch. But by existing, it brings people together with shared interests and I think that that is a huge asset when it comes to even the thought of designing a multi-centre RCT. It’s like you want to design one for [research network].”
  20. 20. Interview ResultsPolicy Factors – Barriers Funding Ethics review process
  21. 21. Interview ResultsPolicy Factors – Barriers“We all tend to want to make the budget as small as we can to increase our chances to actually get it funded and the reality is that some trials really require the full-time effort of somebody who’s got a lot of experience, and therefore comes with a price tag. And it can be hard to make the argument to ensure that you’ve got funding, right? So I think that’s where you start cutting other corners, and you don’t have the data quality, and at the end of the day, you maybe don’t have the rigorous, homerun kind of trial that you had envisioned.”“Most of the problem is to ask for revisions and they are not consistent one between the others. So you can have a question in one and the other one… wants a different answer.”
  22. 22. Interview ResultsPolicy Factors – Facilitators None
  23. 23. Discussion Internal validity is not a primary concern – overshadowed by pragmatics and generalizability  Internal validity is a prerequisite for external validity Lack of formal training and a negative research culture contribute to acceptance of sub-optimal trials Importance of mentorship and clinician-scientists
  24. 24. Future Directions Objectives: To design and evaluate a tailored KT intervention to improve methodological rigor in child health trials. Researcher involvement sought throughout Potential interventions: online modules, checklists
  25. 25. Future Directions Consideration of traditional KT interventions  Adaptation for changing researcher behaviour, rather than clinician behaviour  Social media component Wiki platform  Incorporates successful elements of KT interventions: interactivity, presentation of materials in multiple formats, endorsement by opinion leaders  Informed by Diffusion of Innovations, Social Influences Theory, and Theory of Planned Behaviour
  26. 26. Future Directions
  27. 27. Future Directions
  28. 28. Future Directions
  29. 29. Future Directions
  30. 30. Future Directions
  31. 31. Future Directions
  32. 32. Future DirectionsPilot testing: Trialists, systematic reviewers, trainees Preliminary feedback: content, format, usability Participants will review wiki, take part in a short interview Revisions to wiki based on input
  33. 33. Future Directions RoB wiki to be used as a model for other StaR Child Health SDGs Evaluation of effectiveness  RoB in pediatric trials  Knowledge translation intervention
  34. 34. AcknowledgementsSupervisory Committee: RoB SDG Members:Lisa Hartling Jamie BrehautTerry Klassen An-Wen ChanShannon Scott Jeremy GrimshawDavid Moher Martin Meremikwu Ginny Moyer Prathap Tharyan

×