Exercise 1: PICOTS Population What constitutes an adolescent? What constitutes a diagnosis of autism spectrum disorder (ASD)? Intervention How is transition support defined? Comparator Do we compare to no transition support or directly compare types of support? Outcome What are the goals for adolescents with ASD as they transition to adulthood? Timing How quickly should the outcomes be apparent? Setting Is transition support provided in multiple settings, such as schools, clinics, and the community?
A study that included “individuals over 10 years of age”?
A paper about an intervention for individuals with a range of developmental disabilities, not only autism spectrum disorder (ASD)?
A study of children with Asperger’s syndrome but not other forms of ASD?
Exercise 1: What Would You Do With . . .
There are no “right” answers to these questions.
The team should consider how selection criteria may bias the review or affect applicability.
The team should clearly define how eligibility criteria are operationalized and record these decisions throughout the review.
Exercise 1: No “Right” Answers
What is the efficacy of home uterine activity monitoring for preventing preterm birth among women at low risk of a preterm birth?
Exercise 2: Selecting Criteria for a Narrow Research Question
Efficacy: randomized controlled trials only
Low risk: no previous preterm birth
Yield: 11 studies
Exercise 2: Implications of a Question That Requires Narrow Criteria
What is the best management strategy for overactive bladder (OAB) among women?
OAB is a fairly difficult condition to define.
The study of OAB is often combined with other types of urological conditions, such as stress incontinence or prostate issues, when studied.
OAB is treated pharmacologically, behaviorally, and surgically and with complementary and alternative interventions — each area includes very different types of studies.
Exercise 3: Selecting Criteria for a Broad Research Question
The research team required a minimum sample size N ≥ 50 in each study at study start.
Total excluded for N < 50: 79
Excluded for N < 20: 36
Excluded for N 20 to 29: 23
Excluded for N 30 to 39: 8
Excluded for N 40 to 49: 12
Exercise 3: Impact of Study-Size Criteria on Excluded Studies
Studies had to include at least 75 percent women.
This decision was based on expert opinion and the size and scope of the literature.
Based on this criterion, 40 studies were excluded.
Another 27 studies would have been excluded had the review been limited to studies of only women.
Exercise 3: Impact of Gender on Study Selection Criteria
Case series with N ≥ 2 were included.
Only 3 of 169 studies were randomized controlled trials; 122 were case series.
Because of the relative newness of this area of research, it was important to capture data even from studies without comparison groups.
Example 1: A Maternal-Fetal Surgery Technical Brief
A systematic review on outcomes of cesarean delivery on maternal request.
Conducted for the National Institutes of Health-Office of Medical Applications of Research (NIH-OMAR) State-of-the-Science Conference
Example 2: Systematic Review of Cesarean Delivery
No evidence on outcomes of cesarean delivery on maternal request (CDMR) versus other modes of delivery
Urgent need for actionable evidence
Need to recognize and account for confounders
Example 2: The Challenge in the Systematic Review on Cesarean Delivery
Expand search to include proxies
Weight rungs of evidence to account for confounding
Highest rung: trials of breech delivery, but only for maternal outcomes
Lower rung: planned cesarean versus planned vaginal delivery
Lowest rung: comparisons of maternal and neonatal outcomes from actual modes of delivery
Example 2: The Solution in the Systematic Review on Cesarean Delivery
Specify the scope of the literature to be included
Should be tied to the analytic framework, key questions, and PICOTS
Can reduce bias and support applicability, but run the risk of introducing bias if not chosen appropriately.
Must be applied consistently among individuals on the review team
Need to be recorded throughout the project and in the report for transparency and replicability
Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994;309:1286-91.
Gregoire G, Derderain F, Le L. Selecting the language of the publication included in a meta-analysis: is there a Tower of Babel bias? J Clin Epidemiol 1995;48:159-63.
McAuley L, Pham B, Tugwell P, et al. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 2000;356:1228-31.
McPheeters ML, Briss P, Teutsch SJ, et al. Systematic reviews in public health. In: Brownson RC and Petitti DB, eds. Applied epidemiology . New York: Oxford University Press; 2006. p. 99-124.
Moher D, Pham B, Klassen TP, et al. What contribution do languages other than English make on the results of meta-analyses ? J Clin Epidemiol 2000; 53:964-72.
This presentation was prepared by Melissa L. McPheeters, Ph.D., M.P.H., a member of the Vanderbilt University Evidence-based Practice Center.
The information in this module is currently not included in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare. ahrq.gov/ehc/products/60/294/2009_0805_principles1.pdf).