Laura Koltutsky, profile picture
Uploaded byLaura Koltutsky
PDF, PPTX15,273 views

Introduction to Systematic Reviews

The document provides a comprehensive overview of the systematic review process, detailing its methodology, stages, and importance in evidence-based practice across various fields, particularly in health and social sciences. It emphasizes the need for explicit protocols, rigorous literature searches, inclusion/exclusion criteria, and the assessment of study quality to minimize bias and enhance data reliability. Additionally, it outlines essential tools and resources, including the PRISMA checklist, to facilitate effective documentation and reporting of systematic reviews.

Embed presentation

Download as PDF, PPTX
INTRODUCTION TO THE SYSTEMATIC REVIEW PROCESS Libraries and Cultural Resources Alix Hayden, Laura Koltutsky, Helen Lee Robertson May 7, 2014
Guide Content The content in this guide is adapted from: Tanner, E. E. & Wilson S. J. (2013) Systematic Reviewing for Evidence-Based Practice: An Introductory Workshop. Systematic Reviews: CRD’s guidance for undertaking systematic reviews in health care. Personal expertise, experience, and knowledge of the presenters.
Systematic Review Process Centre for Health Communication and Participation http://navigatingeffectivetreatments.org.au/exploring_systematic_reviews.html
Your Experience 
Pyramid of Evidence In Vitro Research Animal Research Ideas, Editorials, Opinions Case Reports, Case Series Case-Controlled Studies Prospective Cohort Studies Quasi-Experimental Randomized Controlled Double Blind Studies Systematic Reviews, Meta-Analyses
Systematic Reviews Definitions & Descriptions
Types of Reviews
Systematic Review Definitions This methodology prescribes explicit, reproducible, and transparent processes for collating the best available evidence in answer to specific questions. In particular, it requires the use of robust techniques for: searching for and identifying primary studies, selecting the studies to be included in the review, extracting the data from the studies, and appraising the quality of these studies, synthesizing the findings narratively and/or through pooling suitable quantitative data in meta-analysis Dixon-Woods, M., & Sutton, A. (2004). Systematic Review. In Michael S. Lewis-Beck, A. Bryman, & Tim Futing Liao (Eds.), The SAGE Encyclopedia of Social Science Research Methods. (pp. 1111-1112). Thousand Oaks, CA: Sage Publications, Inc. doi: http://dx.doi.org.ezproxy.lib.ucalgary.ca/10.4135/9781412950589.n998
Systematic Reviews / Traditional Reviews Adapted from: Cook, D., Mulrow, C. & Haydens, R. (1997) Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5):376-380. Feature Systematic Review Narrative / Traditional Reviews Question Often focused (clinical) question. Sometimes broad in scope. Sources & Search Explicit search strategy of multiple databases. Comprehensive sources. Not usually specified. Selection Criterion-based selection; uniformly applied Not usually specified. Appraisal Rigorous critical appraisal. Variable. Synthesis Quantitative summary. Also qualitative/narrative. Often qualitative summary. Inferences Based on all available evidence. Based on a sample of the evidence. Grading All evidence is graded (quality) May or may not be graded.
Systematic Review Myths Systematic reviews: are the same as ordinary literature reviews, only bigger include only randomized controlled trials require the adoption of a biomedical model of health are of no relevance outside of health / medicine must involve statistical analysis / synthesis must be conducted by experts can be done without experienced librarian support are not really research Adapted from: Petticrew, M. (2001) Systematic reviews from astronomy to zoology: Myths and misconceptions. BMJ 322:98-101.
Reducing Bias Systematic reviews attempt to minimize bias and error throughout the review process Uses systematic process that is transparent and replicable Document transparent / replicable search strategies Inter-rater reliability on key data Assessment of bias in studies Assessment of study quality Meta-analyis (when possible) to statistically synthesize results across studies
Why Conduct a Systematic Review Rigorous methods to summarize evidence on a given topic
Timeline (Cochrane) Month Activity 1 – 2 Preparation of protocol 3 – 8 Searches for published and unpublished studies 2 – 3 Pilot test of eligibility criteria 3 – 8 Inclusion assessments 3 Pilot test “Risk of Bias” assessment 3 – 10 Validity assessments 3 Pilot test of data collection 3 – 10 Data collection 3 – 10 Data entry 5 – 11 Follow up of missing information 8 – 10 Analysis 1 – 11 Preparation of review report 12 - Keeping the review up-to-date
Systematic Review Organizations Agency for Healthcare Research and Quality Campbell Collaboration (social welfare/education) Centre for Reviews and Dissemination Cochrane Collaboration (health/medicine) EPPI-Centre (social sciences/education) International Initiative for Impact Evaluation Joanna Briggs Institute and many more
Stages of Systematic Review Process Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings Campbell Collaboration
Stage 1: Problem Formulation Stage 2 Data Collection Stage 3 Data Collection Stage 4 Data Analysis Stage 5 Reporting Findings Stage 1 Problem Formulation
Problem Formulation Developing your protocol (research plan) Developing research questions Developing inclusion / exclusion criteria Developing a coding manual for data extraction Determining data management
Research Questions Questions about intervention effects Are juvenile drug courts effective in reducing drug crime recidivism among adolescents, relative to practice as usual? Questions about associations What is the relationship between teacher education/training and student achievement? Does this relationship differ for elementary and secondary school students? Diagnostic/Prognostic questions Which risk assessment (A vs. B) is a better predictor of child maltreatment?
Research Questions Questions often use the PICO(S) framework: Population/Participants Interventions Comparisons Outcomes Study design “To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
Research Questions Questions often use the PICO(S) framework: Population/Participants Interventions Comparisons Outcomes Study design “To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
Research Questions In older adults(P), is an educational intervention(I) more effective than no intervention or an alternative intervention(C) in 1.improving driving awareness/knowledge, 2.improving on-road driving behavior and skills, and, 3.reducing crash rates? (O)
Research Questions In post-cardiac arrest patients with return of spontaneous circulation (P), does therapeutic hypothermia (I) compared with usual care (C), improve morbidity or mortality (O)?
Research Questions Developing a research question for a systematic review requires an understanding of the existing literature, including gaps and uncertainties, definitions, and terminologies
Inclusion / Exclusion Criteria Study eligibility criteria: Clearly defines which subjects or studies will be included in the review Essential for determining relevant and irrelevant studies Determined by the research question MUST be defined before data collection (i.e. conducting the search)
Inclusion / Exclusion Criteria Criteria to consider: Population/ Participants: Age / grade level Gender, ethnicity, socioeconomic status Special or clinical populations Interventions What types of interventions are relevant Identify elements of a program/intervention that are critical Outcomes What is the purpose of the intervention (e.g. quality of life, transition, etc) Study Design What study designs are most appropriate (e.g. RCT)
Inclusion / Exclusion Criteria Criteria to consider: Geographic considerations Time restrictions If limiting the research to specific dates, indicate reason Example: interventions for low birth weight babies – date limit was 1992 because of surfactant trials Language restrictions Remember, it can be difficult to obtain articles in Portuguese even if you are fluent
Inclusion / Exclusion Criteria Objective: Investigate the effectiveness of peer-mediated interventions (PMI) in the treatment of individuals with autism spectrum disorders (ASD)
Inclusion / Exclusion Criteria Note: SPIO is similar to PICO Objective: To determine the effectiveness of mindfulness-based interventions with people diagnosed with multiple sclerosis
Inclusion / Exclusion Criteria
Inclusion / Exclusion Criteria Physical exercise and individuals with autism spectrum disorders (ASD): Objective: review of interventions designed to increase the exercise behavior of individuals with ASD Inclusion Criteria Exclusion Criteria
Coding Manual Data extraction A coding manual to extract data (i.e. study details) from each study needs to be developed prior to collecting data (i.e. searching) Will be discussed under Data Extraction section
Data Management PRISMA – Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews. The PRISMA Statement consists of a 27-item checklist and a four- phase flow diagram.
Data Management PRISMA 27-item checklist http://www.prisma-statement.org/statement.htm
Data Management
Data Management
Data Management
Data Management Bibliographic Management Determine at the beginning of the research planning, what bibliographic management software you will use to manage and track data (i.e. references, citations) Endnote ($) UofC Library supports Refworks (free) UofC Library supports Mendeley (free) UofC Library supports Zotero (free) Reference Manager ($)
Data Management Use bibliographic management software to: Maintain a searchable database of references related to the systematic review Store all references selected for the systematic review Keep track of number of references from each database Determine number of duplicate references Store all discarded references Create citations and bibliography when writing up the results of the SR
Stage 2: Data Collection Stage 1 Problem Formulation Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings Stage 2 Data Collection
Steps in Data Collection The Literature Search Developing inclusion / exclusion criteria Identifying resources to search
The Literature Search The literature search is the most important part of a systematic review (Campbell Collaboration) The literature search is how you obtain your data for your study The goal of the literature search is to find all relevant studies including: Traditional peer reviewed literature Grey literature The literature search must be comprehensive and systematic
The Literature Search The systematic review literature search must: Be well documented Be transparent and reproducible Include a diversity of resources Be iterative
The Literature Search Consult with your subject liaison librarian Document Document Document Databases searched, date range searched, date last searched All search terms for each database, as far as possible Follow the PRISMA (or other) flowchart Number of hits for each database searched Duplicates within databases
The Literature Search Search Strategy Key concepts to be searched (PICOS if using) How are these terms represented in different disciplines? What are related terms / synonyms? Be as comprehensive as possible What are the subject headings (e.g. MeSH) in each database? Review known relevant articles to determine keywords and subject headings Review previous relevant systematic reviews for possible strategy Is there a Cochrane group that has recommended terms
The Literature Search Search Strategy Use truncation nurs* will retrieve: nursing, nurse, nurses, nurses, nursed, etc Use both free text (keywords) and subject headings (i.e. controlled vocabulary), when possible Pilot your search strategy Is your search retrieving the known articles? If no, why not? Check indexing Check truncation Check free text keywords Check descriptors / subject headings It takes a LOT OF TIME
The Literature Search SSRIs Autism Children Selective serotonin reuptake inhibitor* OR Serotonin update Inhibitor OR 5-ht update inhibitor* OR SSRI Autis* OR Infantile autism OR Autistic disorder OR ASD OR Asperger* Child* OR Infan* OR Todder* OR Preschool* Are SSRIs effective for treating autism in children? AND AND
The Literature Search What are the effective pre-graduation interventions (programs, strategies, skill training, etc) aimed at persons with autism spectrum disorders that result in employment in mainstream competitive employment settings? Autism Age Intervention Employment AND AND AND
The Literature Search Save your searches in each database Create an account within the database interface Label the search clearly (is it a draft, final, date, database name) Keep detailed records of all searches conducted: Month, date, year of search Database searched Exact search strategy Any limits used (year of publication, language, etc) Number of hits for each database Remember – goal is to report search strategy in a transparent manner so that it can be replicated
The Literature Search PRESS (Peer Review of Electronic Search Strategies) Translation: Is the search question translated well into search concepts? Operators: Are there any mistakes in the use of Boolean or proximity operators (e.g. ADJ) Subject Headings: are any important subject headings missing or have irrelevant ones been included? Natural language: are any natural language (i.e. free text) terms or spelling variances missing? Is truncation used correctly?
The Literature Search PRESS (Peer Review of Electronic Search Strategies) Spelling & Syntax: does the search strategy have any spelling mistakes, system syntax errors, or wrong line numbers? Limits: Do any of the limits used seem unnecessary or inappropriately used? Adapted for database: has the search strategy been adapted for each database to be searched? It is a good idea to use PRESS with other researchers and your subject liaison librarian to review your search strategy systematically
Identifying Sources Data sources Databases Reference searching Cited bys (forward citations searching, citation pearl growing) Contact with experts Hand searching key journals, journals with poor/no indexing Ongoing studies Grey literature
Databases UofC Library databases: http://library.ucalgary.ca/search- collections/databases-subject-or-name Ask yourself: which disciplines inform my research Educational interventions for young adults with ASD for gaining employment Different interfaces require different searching techniques Review online tutorials Meet with your liaison librarian
Grey Literature Grey literature is defined as: publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
Grey Literature Grey literature is defined as: publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
Grey Literature Grey Literature Includes: Government reports Committee reports Academic papers Theses / dissertations Conference papers & abstracts Conference proceedings Newsletters PPT presentations Discussion reports Program evaluation reports Technical documents Standards / best practice Working papers . . . and more Alberani V., Pietrangeli P. & Mazza A. (1990). The use of grey literature in health sciences: A preliminary survey. Bulletin of the Medical Library Association, 78, 358–363.
Grey Literature Locating Grey Literature: Consider what types of organizations or associations are likely to be interested in the same research question: Government agencies / NGO agencies Government departments Industry, trade, professional organizations Advocacy groups Private agencies
Grey Literature Locating Grey Literature: Canadian Canadian Health Research Collection Canadian Public Policy Collection OpenGrey Resource of grey literature in Europe Grey Literature Report New York Academy of Medicine Dissertations / Theses Proquest Dissertations and Theses Refer to the Systematic Review libguide for more grey literature resource
The Literature Search Interprofessional simulation-based education (IPSE) is becoming an increasingly popular educational strategy worldwide within undergraduate healthcare curricular. The purpose of the literature review was to examine research studies featuring undergraduate IPSE.
The Literature Search
The Literature Search The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer- reviewed literature.
The Literature Search We therefore conducted a systematic review to determine the comparative efficacy of NPWT versus alternate TAC techniques on in-hospital mortality, length of hospital or intensive care unit (ICU) stay, fascial closure rate, and adverse events in critically ill adults with open abdominal wounds.
PRISMA Flowchart
Stage 3: Data Evaluation Stage 1 Problem Formulation Stage 2 Data Collection Stage 4 Data Analysis Stage 5 Reporting Findings Stage 3 Data Evaluation
Data Coding Data extraction is the process by which researchers obtain the necessary information about study characteristics and findings Standardized data extraction provides consistency, thereby potentially reducing bias, improving validity and reliability Ensures that the same information is obtained (extracted) from each study Should be developed early in the project Pilot the data extraction form with a sample of studies Keep track of revisions, corrections, amendments If possible, pilot the exporting, analysis, and outputs of the data extraction form
Data Coding Purpose: Provides an accounting of the research included in your review Identifies key components of the research including interventions, subjects, and methods
Data Coding Levels of Coding Abstract screening Study eligibility screening Study content coding
Abstract Data Coding Abstract Screening Is the study relevant based on the title and abstract Apply title/ abstract eligibility criteria, which are often less specific than the full inclusion/exclusion criteria If yes, retrieve full text If unclear, retrieve full text Exclude obviously irrelevant studies Include two trained raters working independently if possible
Full-Text Data Coding Full-Text Eligibility Screening Develop an eligibility screening form based on the inclusion / exclusion criteria Complete for each study retrieved that is potentially eligible Include reasons for ineligible studies (for PRISMA reporting)
Study Content Coding Data extraction form: Be consistent in the order and style you use to describe the information from each study Provide instructions, decision rules, and training on the use of the data extraction form Practice using the data extraction form Compare completed pilot forms Ensure that there is consistency in extracting data Ideally two researchers should independently perform data extraction Record any missing information as unclear or not described Makes it clear that the information was not included in the study rather than you forgot to extract it
Study Content Coding Systemtaic reviews: CRD’s guidance for undertaking systematic reviews in health care. 2009. Retrieved from: https://www.york.ac.uk/inst/crd/index_guidance.htm
Study Content Coding Cochrane has developed templates that can be revised Google: “data extraction form” cochrane Review published systematic reviews relevant to your research area Request permission to adapt the data extraction form
Study Content Coding Software for data extraction RevMan (Cochrane – can use free of charge for academic purposes) EPPI-Reviewer (one month free trial) JBI Sumari (subscription based) NVivo (UofC site license) Abstraktr Excel
Stage 4: Data Analysis Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 5 Reporting Findings Stage 4 Data Analysis
Data Analysis Planned analysis of included studies: Meta-analysis and/or narrative analysis
Data Analysis Analyzing the Evidence Each included study is analyzed for bias and quality Can be component of data extraction form Cochrane risk of bias framework GRADE system Method quality / critical appraisal checklists Quantitative Qualitative Critical appraisal resources: http://fhswedge.csu.mcmaster.ca/cepftp/qasite/CriticalAppraisal.html
Stage 5: Reporting Findings Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings
Reporting Findings
Reporting Findings If you plan on publishing in a journal Review the intended journal for specific guidelines for systematic reviews BEFORE starting your review If there are no guidelines, review other systematic reviews in the intended journal
Checklists Refer to the PRISMA checklist How does your systematic review stand up? Review: Appraisal of a systematic review using a checklist: Notes and example 1 Clinical Evidence (BMJ) http://clinicalevidence.bmj.com/x/set/static/ebm/toolbox/665052.html
Alix Hayden KNES, NURS, EDUC, CORE ahayden@ucalgary.ca Laura Koltutsky PSYC, SOWK, Sociology ljkoltut@ucalgary.ca Helen Lee Robertson Medicine helen.lee.robertson@ucalgary.ca Contact:

More Related Content

PPT
Systematic Review: Beginner's Guide
bySaee Deshpamde
 
PDF
Introduction to Systematic Review & Meta-Analysis
byHasanain Ghazi
 
PPTX
Systematic review
byKhalid Mahmood
 
PPT
Systematic review
byHesham Gaber
 
PPTX
Systematic reviews
byZera Day
 
PPTX
Systematic review and meta analaysis course - part 1
byAhmed Negida
 
PDF
Introduction to systematic reviews
byOmar Midani
 
PPT
Meta-analysis and systematic reviews
bycoolboy101pk
 
Systematic Review: Beginner's Guide
bySaee Deshpamde
 
Introduction to Systematic Review & Meta-Analysis
byHasanain Ghazi
 
Systematic review
byKhalid Mahmood
 
Systematic review
byHesham Gaber
 
Systematic reviews
byZera Day
 
Systematic review and meta analaysis course - part 1
byAhmed Negida
 
Introduction to systematic reviews
byOmar Midani
 
Meta-analysis and systematic reviews
bycoolboy101pk
 

What's hot

PDF
Systematic review
byAboubakr Elnashar
 
PPTX
Systematic Literature Review and Meta-Analysis (1).pptx
byEDCEILIYAHDENTALCARE
 
PPTX
Different Types of health research
byDr Ghaiath Hussein
 
PPT
Systematic Reviews: the process, quantitative, qualitative and mixed methods ...
byhealthlibaust2012
 
PPTX
Systematic Review & Meta-Analysis Course - Summary Slides
byKing Abdullah Medical City (KAMC) Research Center
 
PPTX
Bias in clinical research
bysathyanarayanan varadarajan
 
PPTX
Systematic Review
by2015UPM
 
PPTX
Systematic review and meta analaysis course - part 2
byAhmed Negida
 
PPT
Systematic review ppt
byBasil Asay
 
PPTX
Literature search techniques
byAhmed Elfaitury
 
PPTX
Basics of Systematic Review and Meta-analysis: Part 2
byRizwan S A
 
PPTX
systematic review and metaanalysis
byDrSridevi NH
 
PPTX
Overview of systematic review and meta analysis
byDrsnehas2
 
PPTX
Study design in research
byKusum Gaur
 
PPTX
Introduction to meta-analysis (1612_MA_workshop)
byAhmed Negida
 
PPTX
Systematic Review & Meta Analysis.pptx
byDr. Anik Chakraborty
 
PPTX
An introduction to qualitative research
byNajibullah Safi
 
PPTX
Qualitative Research Methods for Public Health
byCelia Emmelhainz
 
PDF
Introduction to Systematic Reviews and Meta-analyses-ACSRM
bySystematic Reviews Network (SRN)
 
PPT
Hierarchies of Evidence 2017
byLucia Ravi
 
Systematic review
byAboubakr Elnashar
 
Systematic Literature Review and Meta-Analysis (1).pptx
byEDCEILIYAHDENTALCARE
 
Different Types of health research
byDr Ghaiath Hussein
 
Systematic Reviews: the process, quantitative, qualitative and mixed methods ...
byhealthlibaust2012
 
Systematic Review & Meta-Analysis Course - Summary Slides
byKing Abdullah Medical City (KAMC) Research Center
 
Bias in clinical research
bysathyanarayanan varadarajan
 
Systematic Review
by2015UPM
 
Systematic review and meta analaysis course - part 2
byAhmed Negida
 
Systematic review ppt
byBasil Asay
 
Literature search techniques
byAhmed Elfaitury
 
Basics of Systematic Review and Meta-analysis: Part 2
byRizwan S A
 
systematic review and metaanalysis
byDrSridevi NH
 
Overview of systematic review and meta analysis
byDrsnehas2
 
Study design in research
byKusum Gaur
 
Introduction to meta-analysis (1612_MA_workshop)
byAhmed Negida
 
Systematic Review & Meta Analysis.pptx
byDr. Anik Chakraborty
 
An introduction to qualitative research
byNajibullah Safi
 
Qualitative Research Methods for Public Health
byCelia Emmelhainz
 
Introduction to Systematic Reviews and Meta-analyses-ACSRM
bySystematic Reviews Network (SRN)
 
Hierarchies of Evidence 2017
byLucia Ravi
 

Similar to Introduction to Systematic Reviews

PPTX
Systematic Reviews: A Comprehensive Guide
byDharun Sao
 
PPTX
Meta analysis.pptx
bywondesengemechu
 
PPTX
Systematic review and meta analysis.pptx
byAbasAhmed7
 
PPTX
SystematicreviewJan2017.pptx
byMarwan98
 
PPT
Edmedia2011 lit.sys.review
byValerie Irvine
 
PPTX
developingrobusteligibilitycriteriaandanefficientstudypresentation-2504171232...
byDoreenNampamya1
 
PPTX
Developing Robust Eligibility Criteria and an Efficient Study - Dr Leonard Uz...
bySystematic Reviews Network (SRN)
 
PPTX
systematic review and meta-analysis.pptx
byVinothini Settu
 
PPT
review of Litt on clinical research and Methodology
byDr.Venkata Suresh Ponnuru
 
PPT
مراجعة الأدبيات المنهجيةsystematic literature review .ppt
by901202
 
PPT
MA Talk Temple.ppt
byShamazMohamed
 
PPTX
SYSTEMATIC REVIEW: A STEP-BY-STEP APPROACH
bykabiruabubakar3
 
PPTX
Understanding the process of doing a systematic review
byAmizhthan Sammanthane
 
PPTX
An Introduction to writing an Systematic review of literature - Scientific re...
byPubrica
 
PPTX
Systematic review my presentation.pptx
bynehalabosamra91
 
PPT
An introduction to conducting a systematic literature review for social scien...
byrosie.dunne
 
PDF
Formulating the Topic and Research Questions for Systematic Reviews - Dr Carl...
bySystematic Reviews Network (SRN)
 
PDF
Retrospective versus | Meta analysis | Systematic literature review
byPubrica
 
PDF
Doing a Systematic Review A Student s Guide 2nd ed 2nd Edition Angela Boland
byyasijandic
 
PPTX
Systematic literature review | Meta analysis | Retrospective versus
byPubrica
 
Systematic Reviews: A Comprehensive Guide
byDharun Sao
 
Meta analysis.pptx
bywondesengemechu
 
Systematic review and meta analysis.pptx
byAbasAhmed7
 
SystematicreviewJan2017.pptx
byMarwan98
 
Edmedia2011 lit.sys.review
byValerie Irvine
 
developingrobusteligibilitycriteriaandanefficientstudypresentation-2504171232...
byDoreenNampamya1
 
Developing Robust Eligibility Criteria and an Efficient Study - Dr Leonard Uz...
bySystematic Reviews Network (SRN)
 
systematic review and meta-analysis.pptx
byVinothini Settu
 
review of Litt on clinical research and Methodology
byDr.Venkata Suresh Ponnuru
 
مراجعة الأدبيات المنهجيةsystematic literature review .ppt
by901202
 
MA Talk Temple.ppt
byShamazMohamed
 
SYSTEMATIC REVIEW: A STEP-BY-STEP APPROACH
bykabiruabubakar3
 
Understanding the process of doing a systematic review
byAmizhthan Sammanthane
 
An Introduction to writing an Systematic review of literature - Scientific re...
byPubrica
 
Systematic review my presentation.pptx
bynehalabosamra91
 
An introduction to conducting a systematic literature review for social scien...
byrosie.dunne
 
Formulating the Topic and Research Questions for Systematic Reviews - Dr Carl...
bySystematic Reviews Network (SRN)
 
Retrospective versus | Meta analysis | Systematic literature review
byPubrica
 
Doing a Systematic Review A Student s Guide 2nd ed 2nd Edition Angela Boland
byyasijandic
 
Systematic literature review | Meta analysis | Retrospective versus
byPubrica
 

Recently uploaded

PPTX
GRADE 8 - QUARTER 4 TYPES AND PARTS OF LETTER.pptx
byvillariasjazminercai
 
PPTX
Math 8 Quarter 4 Week 3 PRIMARY DATA.pptx
byshahanieabbat3
 
PPTX
Guidelines for reporting social networks and personal networks data
byisidromj
 
PDF
Chapter 05 Drugs Acting on the Central Nervous System: Anticonvulsant (Antiep...
bySandeshSul
 
PDF
Why Projects Fail – The Need to “Do the Right Project” and “Do the Project Ri...
byAssociation for Project Management
 
PDF
Artificial Intelligence in Research and Academic Writing, Workshop on Researc...
byProf. Vinod Kumar Kanvaria
 
PDF
RAJAT ARORA SIR PHYSICAL EDUCATION NOTES ALL CHAPTERS .pdf
bysumitkannoujiya74
 
PPTX
West Hatch High School - GCSE Media Studies
byWestHatch
 
PPTX
A detailed notes on Conjugate Vaccine and its mechanism.
byDr. R. Ram
 
PPTX
How to Easily Track Appraisal Analysis Report in Odoo 18 Appraisal
byCeline George
 
PDF
APPSC APPSC AEE-AE GENERAL STUDIES QUESTION PAPER.pdf
byssuser9d8e4e
 
PDF
Tetracycline Class of Antibiotics: SAR, MOA and Uses
bymominzarinamdnajeeb
 
PPTX
Chapter 1: Introduction to Economics - Macroeconomics and Microeconomics.pptx
byFJ M
 
PPTX
West Hatch High School -- GCSE Geography
byWestHatch
 
PDF
Using T-Test to Analyze Research Data.pdf
byThelma Villaflores
 
PPTX
industrial arts grade 6 tle_q4_w3_melc.pptx
byadrianatillaga1
 
PPTX
Modeling Simple Equation using Bar Model.pptx
byshahanieabbat3
 
PDF
MP High Court Stenographer (Grade 3) Previous Year's Question Paper's Compute...
bySONU HEETSON
 
PPTX
DRYING Pharmaceutical engineering Semester III 2nd year B Pharmacy
byChetana Wagh
 
PPT
West Hatch High School - GCSE History Option
byWestHatch
 
GRADE 8 - QUARTER 4 TYPES AND PARTS OF LETTER.pptx
byvillariasjazminercai
 
Math 8 Quarter 4 Week 3 PRIMARY DATA.pptx
byshahanieabbat3
 
Guidelines for reporting social networks and personal networks data
byisidromj
 
Chapter 05 Drugs Acting on the Central Nervous System: Anticonvulsant (Antiep...
bySandeshSul
 
Why Projects Fail – The Need to “Do the Right Project” and “Do the Project Ri...
byAssociation for Project Management
 
Artificial Intelligence in Research and Academic Writing, Workshop on Researc...
byProf. Vinod Kumar Kanvaria
 
RAJAT ARORA SIR PHYSICAL EDUCATION NOTES ALL CHAPTERS .pdf
bysumitkannoujiya74
 
West Hatch High School - GCSE Media Studies
byWestHatch
 
A detailed notes on Conjugate Vaccine and its mechanism.
byDr. R. Ram
 
How to Easily Track Appraisal Analysis Report in Odoo 18 Appraisal
byCeline George
 
APPSC APPSC AEE-AE GENERAL STUDIES QUESTION PAPER.pdf
byssuser9d8e4e
 
Tetracycline Class of Antibiotics: SAR, MOA and Uses
bymominzarinamdnajeeb
 
Chapter 1: Introduction to Economics - Macroeconomics and Microeconomics.pptx
byFJ M
 
West Hatch High School -- GCSE Geography
byWestHatch
 
Using T-Test to Analyze Research Data.pdf
byThelma Villaflores
 
industrial arts grade 6 tle_q4_w3_melc.pptx
byadrianatillaga1
 
Modeling Simple Equation using Bar Model.pptx
byshahanieabbat3
 
MP High Court Stenographer (Grade 3) Previous Year's Question Paper's Compute...
bySONU HEETSON
 
DRYING Pharmaceutical engineering Semester III 2nd year B Pharmacy
byChetana Wagh
 
West Hatch High School - GCSE History Option
byWestHatch
 

Introduction to Systematic Reviews

  • 1.
    INTRODUCTION TO THESYSTEMATIC REVIEW PROCESS Libraries and Cultural Resources Alix Hayden, Laura Koltutsky, Helen Lee Robertson May 7, 2014
  • 2.
    Guide Content Thecontent in this guide is adapted from: Tanner, E. E. & Wilson S. J. (2013) Systematic Reviewing for Evidence-Based Practice: An Introductory Workshop. Systematic Reviews: CRD’s guidance for undertaking systematic reviews in health care. Personal expertise, experience, and knowledge of the presenters.
  • 3.
    Systematic Review Process Centre for Health Communication and Participation http://navigatingeffectivetreatments.org.au/exploring_systematic_reviews.html
  • 4.
  • 5.
    Pyramid of Evidence In Vitro Research Animal Research Ideas, Editorials, Opinions Case Reports, Case Series Case-Controlled Studies Prospective Cohort Studies Quasi-Experimental Randomized Controlled Double Blind Studies Systematic Reviews, Meta-Analyses
  • 6.
  • 7.
  • 8.
    Systematic Review Definitions This methodology prescribes explicit, reproducible, and transparent processes for collating the best available evidence in answer to specific questions. In particular, it requires the use of robust techniques for: searching for and identifying primary studies, selecting the studies to be included in the review, extracting the data from the studies, and appraising the quality of these studies, synthesizing the findings narratively and/or through pooling suitable quantitative data in meta-analysis Dixon-Woods, M., & Sutton, A. (2004). Systematic Review. In Michael S. Lewis-Beck, A. Bryman, & Tim Futing Liao (Eds.), The SAGE Encyclopedia of Social Science Research Methods. (pp. 1111-1112). Thousand Oaks, CA: Sage Publications, Inc. doi: http://dx.doi.org.ezproxy.lib.ucalgary.ca/10.4135/9781412950589.n998
  • 9.
    Systematic Reviews /Traditional Reviews Adapted from: Cook, D., Mulrow, C. & Haydens, R. (1997) Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5):376-380. Feature Systematic Review Narrative / Traditional Reviews Question Often focused (clinical) question. Sometimes broad in scope. Sources & Search Explicit search strategy of multiple databases. Comprehensive sources. Not usually specified. Selection Criterion-based selection; uniformly applied Not usually specified. Appraisal Rigorous critical appraisal. Variable. Synthesis Quantitative summary. Also qualitative/narrative. Often qualitative summary. Inferences Based on all available evidence. Based on a sample of the evidence. Grading All evidence is graded (quality) May or may not be graded.
  • 10.
    Systematic Review Myths Systematic reviews: are the same as ordinary literature reviews, only bigger include only randomized controlled trials require the adoption of a biomedical model of health are of no relevance outside of health / medicine must involve statistical analysis / synthesis must be conducted by experts can be done without experienced librarian support are not really research Adapted from: Petticrew, M. (2001) Systematic reviews from astronomy to zoology: Myths and misconceptions. BMJ 322:98-101.
  • 11.
    Reducing Bias Systematicreviews attempt to minimize bias and error throughout the review process Uses systematic process that is transparent and replicable Document transparent / replicable search strategies Inter-rater reliability on key data Assessment of bias in studies Assessment of study quality Meta-analyis (when possible) to statistically synthesize results across studies
  • 12.
    Why Conduct aSystematic Review Rigorous methods to summarize evidence on a given topic
  • 13.
    Timeline (Cochrane) Month Activity 1 – 2 Preparation of protocol 3 – 8 Searches for published and unpublished studies 2 – 3 Pilot test of eligibility criteria 3 – 8 Inclusion assessments 3 Pilot test “Risk of Bias” assessment 3 – 10 Validity assessments 3 Pilot test of data collection 3 – 10 Data collection 3 – 10 Data entry 5 – 11 Follow up of missing information 8 – 10 Analysis 1 – 11 Preparation of review report 12 - Keeping the review up-to-date
  • 14.
    Systematic Review Organizations Agency for Healthcare Research and Quality Campbell Collaboration (social welfare/education) Centre for Reviews and Dissemination Cochrane Collaboration (health/medicine) EPPI-Centre (social sciences/education) International Initiative for Impact Evaluation Joanna Briggs Institute and many more
  • 15.
    Stages of SystematicReview Process Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings Campbell Collaboration
  • 16.
    Stage 1: ProblemFormulation Stage 2 Data Collection Stage 3 Data Collection Stage 4 Data Analysis Stage 5 Reporting Findings Stage 1 Problem Formulation
  • 17.
    Problem Formulation Developingyour protocol (research plan) Developing research questions Developing inclusion / exclusion criteria Developing a coding manual for data extraction Determining data management
  • 18.
    Research Questions Questionsabout intervention effects Are juvenile drug courts effective in reducing drug crime recidivism among adolescents, relative to practice as usual? Questions about associations What is the relationship between teacher education/training and student achievement? Does this relationship differ for elementary and secondary school students? Diagnostic/Prognostic questions Which risk assessment (A vs. B) is a better predictor of child maltreatment?
  • 19.
    Research Questions Questionsoften use the PICO(S) framework: Population/Participants Interventions Comparisons Outcomes Study design “To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
  • 20.
    Research Questions Questionsoften use the PICO(S) framework: Population/Participants Interventions Comparisons Outcomes Study design “To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
  • 21.
    Research Questions Inolder adults(P), is an educational intervention(I) more effective than no intervention or an alternative intervention(C) in 1.improving driving awareness/knowledge, 2.improving on-road driving behavior and skills, and, 3.reducing crash rates? (O)
  • 22.
    Research Questions Inpost-cardiac arrest patients with return of spontaneous circulation (P), does therapeutic hypothermia (I) compared with usual care (C), improve morbidity or mortality (O)?
  • 23.
    Research Questions Developinga research question for a systematic review requires an understanding of the existing literature, including gaps and uncertainties, definitions, and terminologies
  • 24.
    Inclusion / ExclusionCriteria Study eligibility criteria: Clearly defines which subjects or studies will be included in the review Essential for determining relevant and irrelevant studies Determined by the research question MUST be defined before data collection (i.e. conducting the search)
  • 25.
    Inclusion / ExclusionCriteria Criteria to consider: Population/ Participants: Age / grade level Gender, ethnicity, socioeconomic status Special or clinical populations Interventions What types of interventions are relevant Identify elements of a program/intervention that are critical Outcomes What is the purpose of the intervention (e.g. quality of life, transition, etc) Study Design What study designs are most appropriate (e.g. RCT)
  • 26.
    Inclusion / ExclusionCriteria Criteria to consider: Geographic considerations Time restrictions If limiting the research to specific dates, indicate reason Example: interventions for low birth weight babies – date limit was 1992 because of surfactant trials Language restrictions Remember, it can be difficult to obtain articles in Portuguese even if you are fluent
  • 27.
    Inclusion / ExclusionCriteria Objective: Investigate the effectiveness of peer-mediated interventions (PMI) in the treatment of individuals with autism spectrum disorders (ASD)
  • 28.
    Inclusion / ExclusionCriteria Note: SPIO is similar to PICO Objective: To determine the effectiveness of mindfulness-based interventions with people diagnosed with multiple sclerosis
  • 29.
  • 30.
    Inclusion / ExclusionCriteria Physical exercise and individuals with autism spectrum disorders (ASD): Objective: review of interventions designed to increase the exercise behavior of individuals with ASD Inclusion Criteria Exclusion Criteria
  • 31.
    Coding Manual Dataextraction A coding manual to extract data (i.e. study details) from each study needs to be developed prior to collecting data (i.e. searching) Will be discussed under Data Extraction section
  • 32.
    Data Management PRISMA– Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews. The PRISMA Statement consists of a 27-item checklist and a four- phase flow diagram.
  • 33.
    Data Management PRISMA27-item checklist http://www.prisma-statement.org/statement.htm
  • 34.
  • 35.
  • 36.
  • 37.
    Data Management BibliographicManagement Determine at the beginning of the research planning, what bibliographic management software you will use to manage and track data (i.e. references, citations) Endnote ($) UofC Library supports Refworks (free) UofC Library supports Mendeley (free) UofC Library supports Zotero (free) Reference Manager ($)
  • 38.
    Data Management Usebibliographic management software to: Maintain a searchable database of references related to the systematic review Store all references selected for the systematic review Keep track of number of references from each database Determine number of duplicate references Store all discarded references Create citations and bibliography when writing up the results of the SR
  • 39.
    Stage 2: DataCollection Stage 1 Problem Formulation Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings Stage 2 Data Collection
  • 40.
    Steps in DataCollection The Literature Search Developing inclusion / exclusion criteria Identifying resources to search
  • 41.
    The Literature Search The literature search is the most important part of a systematic review (Campbell Collaboration) The literature search is how you obtain your data for your study The goal of the literature search is to find all relevant studies including: Traditional peer reviewed literature Grey literature The literature search must be comprehensive and systematic
  • 42.
    The Literature Search The systematic review literature search must: Be well documented Be transparent and reproducible Include a diversity of resources Be iterative
  • 43.
    The Literature Search Consult with your subject liaison librarian Document Document Document Databases searched, date range searched, date last searched All search terms for each database, as far as possible Follow the PRISMA (or other) flowchart Number of hits for each database searched Duplicates within databases
  • 44.
    The Literature Search Search Strategy Key concepts to be searched (PICOS if using) How are these terms represented in different disciplines? What are related terms / synonyms? Be as comprehensive as possible What are the subject headings (e.g. MeSH) in each database? Review known relevant articles to determine keywords and subject headings Review previous relevant systematic reviews for possible strategy Is there a Cochrane group that has recommended terms
  • 45.
    The Literature Search Search Strategy Use truncation nurs* will retrieve: nursing, nurse, nurses, nurses, nursed, etc Use both free text (keywords) and subject headings (i.e. controlled vocabulary), when possible Pilot your search strategy Is your search retrieving the known articles? If no, why not? Check indexing Check truncation Check free text keywords Check descriptors / subject headings It takes a LOT OF TIME
  • 46.
    The Literature Search SSRIs Autism Children Selective serotonin reuptake inhibitor* OR Serotonin update Inhibitor OR 5-ht update inhibitor* OR SSRI Autis* OR Infantile autism OR Autistic disorder OR ASD OR Asperger* Child* OR Infan* OR Todder* OR Preschool* Are SSRIs effective for treating autism in children? AND AND
  • 47.
    The Literature Search What are the effective pre-graduation interventions (programs, strategies, skill training, etc) aimed at persons with autism spectrum disorders that result in employment in mainstream competitive employment settings? Autism Age Intervention Employment AND AND AND
  • 48.
    The Literature Search Save your searches in each database Create an account within the database interface Label the search clearly (is it a draft, final, date, database name) Keep detailed records of all searches conducted: Month, date, year of search Database searched Exact search strategy Any limits used (year of publication, language, etc) Number of hits for each database Remember – goal is to report search strategy in a transparent manner so that it can be replicated
  • 49.
    The Literature Search PRESS (Peer Review of Electronic Search Strategies) Translation: Is the search question translated well into search concepts? Operators: Are there any mistakes in the use of Boolean or proximity operators (e.g. ADJ) Subject Headings: are any important subject headings missing or have irrelevant ones been included? Natural language: are any natural language (i.e. free text) terms or spelling variances missing? Is truncation used correctly?
  • 50.
    The Literature Search PRESS (Peer Review of Electronic Search Strategies) Spelling & Syntax: does the search strategy have any spelling mistakes, system syntax errors, or wrong line numbers? Limits: Do any of the limits used seem unnecessary or inappropriately used? Adapted for database: has the search strategy been adapted for each database to be searched? It is a good idea to use PRESS with other researchers and your subject liaison librarian to review your search strategy systematically
  • 51.
    Identifying Sources Datasources Databases Reference searching Cited bys (forward citations searching, citation pearl growing) Contact with experts Hand searching key journals, journals with poor/no indexing Ongoing studies Grey literature
  • 52.
    Databases UofC Librarydatabases: http://library.ucalgary.ca/search- collections/databases-subject-or-name Ask yourself: which disciplines inform my research Educational interventions for young adults with ASD for gaining employment Different interfaces require different searching techniques Review online tutorials Meet with your liaison librarian
  • 53.
    Grey Literature Greyliterature is defined as: publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
  • 54.
    Grey Literature Greyliterature is defined as: publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
  • 55.
    Grey Literature GreyLiterature Includes: Government reports Committee reports Academic papers Theses / dissertations Conference papers & abstracts Conference proceedings Newsletters PPT presentations Discussion reports Program evaluation reports Technical documents Standards / best practice Working papers . . . and more Alberani V., Pietrangeli P. & Mazza A. (1990). The use of grey literature in health sciences: A preliminary survey. Bulletin of the Medical Library Association, 78, 358–363.
  • 56.
    Grey Literature LocatingGrey Literature: Consider what types of organizations or associations are likely to be interested in the same research question: Government agencies / NGO agencies Government departments Industry, trade, professional organizations Advocacy groups Private agencies
  • 57.
    Grey Literature LocatingGrey Literature: Canadian Canadian Health Research Collection Canadian Public Policy Collection OpenGrey Resource of grey literature in Europe Grey Literature Report New York Academy of Medicine Dissertations / Theses Proquest Dissertations and Theses Refer to the Systematic Review libguide for more grey literature resource
  • 58.
    The Literature Search Interprofessional simulation-based education (IPSE) is becoming an increasingly popular educational strategy worldwide within undergraduate healthcare curricular. The purpose of the literature review was to examine research studies featuring undergraduate IPSE.
  • 59.
  • 60.
    The Literature Search The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer- reviewed literature.
  • 61.
    The Literature Search We therefore conducted a systematic review to determine the comparative efficacy of NPWT versus alternate TAC techniques on in-hospital mortality, length of hospital or intensive care unit (ICU) stay, fascial closure rate, and adverse events in critically ill adults with open abdominal wounds.
  • 62.
  • 63.
    Stage 3: DataEvaluation Stage 1 Problem Formulation Stage 2 Data Collection Stage 4 Data Analysis Stage 5 Reporting Findings Stage 3 Data Evaluation
  • 64.
    Data Coding Dataextraction is the process by which researchers obtain the necessary information about study characteristics and findings Standardized data extraction provides consistency, thereby potentially reducing bias, improving validity and reliability Ensures that the same information is obtained (extracted) from each study Should be developed early in the project Pilot the data extraction form with a sample of studies Keep track of revisions, corrections, amendments If possible, pilot the exporting, analysis, and outputs of the data extraction form
  • 65.
    Data Coding Purpose: Provides an accounting of the research included in your review Identifies key components of the research including interventions, subjects, and methods
  • 66.
    Data Coding Levelsof Coding Abstract screening Study eligibility screening Study content coding
  • 67.
    Abstract Data Coding Abstract Screening Is the study relevant based on the title and abstract Apply title/ abstract eligibility criteria, which are often less specific than the full inclusion/exclusion criteria If yes, retrieve full text If unclear, retrieve full text Exclude obviously irrelevant studies Include two trained raters working independently if possible
  • 68.
    Full-Text Data Coding Full-Text Eligibility Screening Develop an eligibility screening form based on the inclusion / exclusion criteria Complete for each study retrieved that is potentially eligible Include reasons for ineligible studies (for PRISMA reporting)
  • 69.
    Study Content Coding Data extraction form: Be consistent in the order and style you use to describe the information from each study Provide instructions, decision rules, and training on the use of the data extraction form Practice using the data extraction form Compare completed pilot forms Ensure that there is consistency in extracting data Ideally two researchers should independently perform data extraction Record any missing information as unclear or not described Makes it clear that the information was not included in the study rather than you forgot to extract it
  • 70.
    Study Content Coding Systemtaic reviews: CRD’s guidance for undertaking systematic reviews in health care. 2009. Retrieved from: https://www.york.ac.uk/inst/crd/index_guidance.htm
  • 71.
    Study Content Coding Cochrane has developed templates that can be revised Google: “data extraction form” cochrane Review published systematic reviews relevant to your research area Request permission to adapt the data extraction form
  • 72.
    Study Content Coding Software for data extraction RevMan (Cochrane – can use free of charge for academic purposes) EPPI-Reviewer (one month free trial) JBI Sumari (subscription based) NVivo (UofC site license) Abstraktr Excel
  • 73.
    Stage 4: DataAnalysis Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 5 Reporting Findings Stage 4 Data Analysis
  • 74.
    Data Analysis Plannedanalysis of included studies: Meta-analysis and/or narrative analysis
  • 75.
    Data Analysis Analyzingthe Evidence Each included study is analyzed for bias and quality Can be component of data extraction form Cochrane risk of bias framework GRADE system Method quality / critical appraisal checklists Quantitative Qualitative Critical appraisal resources: http://fhswedge.csu.mcmaster.ca/cepftp/qasite/CriticalAppraisal.html
  • 76.
    Stage 5: ReportingFindings Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings
  • 77.
  • 78.
    Reporting Findings Ifyou plan on publishing in a journal Review the intended journal for specific guidelines for systematic reviews BEFORE starting your review If there are no guidelines, review other systematic reviews in the intended journal
  • 79.
    Checklists Refer tothe PRISMA checklist How does your systematic review stand up? Review: Appraisal of a systematic review using a checklist: Notes and example 1 Clinical Evidence (BMJ) http://clinicalevidence.bmj.com/x/set/static/ebm/toolbox/665052.html
  • 80.
    Alix Hayden KNES,NURS, EDUC, CORE ahayden@ucalgary.ca Laura Koltutsky PSYC, SOWK, Sociology ljkoltut@ucalgary.ca Helen Lee Robertson Medicine helen.lee.robertson@ucalgary.ca Contact: