SlideShare a Scribd company logo
1 of 80
Download to read offline
INTRODUCTION TO THE SYSTEMATIC REVIEW PROCESS 
Libraries and Cultural Resources 
Alix Hayden, Laura Koltutsky, Helen Lee Robertson May 7, 2014
Guide Content 
The content in this guide is adapted from: 
Tanner, E. E. & Wilson S. J. (2013) Systematic Reviewing for Evidence-Based Practice: An Introductory Workshop. 
Systematic Reviews: CRD’s guidance for undertaking systematic reviews in health care. 
Personal expertise, experience, and knowledge of the presenters.
Systematic Review Process 
Centre for Health Communication and Participation 
http://navigatingeffectivetreatments.org.au/exploring_systematic_reviews.html
Your Experience 

Pyramid of Evidence 
In Vitro Research 
Animal Research 
Ideas, Editorials, Opinions 
Case Reports, Case Series 
Case-Controlled Studies 
Prospective Cohort Studies 
Quasi-Experimental 
Randomized Controlled Double Blind Studies 
Systematic Reviews, Meta-Analyses
Systematic Reviews Definitions & Descriptions
Types of Reviews
Systematic Review Definitions 
This methodology prescribes explicit, reproducible, and transparent processes for collating the best available evidence in answer to specific questions. 
In particular, it requires the use of robust techniques for: 
searching for and identifying primary studies, 
selecting the studies to be included in the review, 
extracting the data from the studies, and 
appraising the quality of these studies, 
synthesizing the findings narratively and/or through pooling suitable quantitative data in meta-analysis 
Dixon-Woods, M., & Sutton, A. (2004). Systematic Review. In Michael S. Lewis-Beck, A. Bryman, & Tim Futing Liao (Eds.), The SAGE Encyclopedia of Social Science Research Methods. (pp. 1111-1112). Thousand Oaks, CA: Sage Publications, Inc. doi: http://dx.doi.org.ezproxy.lib.ucalgary.ca/10.4135/9781412950589.n998
Systematic Reviews / Traditional Reviews 
Adapted from: Cook, D., Mulrow, C. & Haydens, R. (1997) Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5):376-380. 
Feature 
Systematic Review 
Narrative / Traditional Reviews 
Question 
Often focused (clinical) question. 
Sometimes broad in scope. 
Sources & Search 
Explicit search strategy of multiple databases. Comprehensive sources. 
Not usually specified. 
Selection 
Criterion-based selection; uniformly applied 
Not usually specified. 
Appraisal 
Rigorous critical appraisal. 
Variable. 
Synthesis 
Quantitative summary. Also qualitative/narrative. 
Often qualitative summary. 
Inferences 
Based on all available evidence. 
Based on a sample of the evidence. 
Grading 
All evidence is graded (quality) 
May or may not be graded.
Systematic Review Myths 
Systematic reviews: 
are the same as ordinary literature reviews, only bigger 
include only randomized controlled trials 
require the adoption of a biomedical model of health 
are of no relevance outside of health / medicine 
must involve statistical analysis / synthesis 
must be conducted by experts 
can be done without experienced librarian support 
are not really research 
Adapted from: Petticrew, M. (2001) Systematic reviews from astronomy to zoology: Myths and misconceptions. BMJ 322:98-101.
Reducing Bias 
Systematic reviews attempt to minimize bias and error throughout the review process 
Uses systematic process that is transparent and replicable 
Document transparent / replicable search strategies 
Inter-rater reliability on key data 
Assessment of bias in studies 
Assessment of study quality 
Meta-analyis (when possible) to statistically synthesize results across studies
Why Conduct a Systematic Review 
Rigorous methods to summarize evidence on a given topic
Timeline (Cochrane) 
Month 
Activity 
1 – 2 
Preparation of protocol 
3 – 8 
Searches for published and unpublished studies 
2 – 3 
Pilot test of eligibility criteria 
3 – 8 
Inclusion assessments 
3 
Pilot test “Risk of Bias” assessment 
3 – 10 
Validity assessments 
3 
Pilot test of data collection 
3 – 10 
Data collection 
3 – 10 
Data entry 
5 – 11 
Follow up of missing information 
8 – 10 
Analysis 
1 – 11 
Preparation of review report 
12 - 
Keeping the review up-to-date
Systematic Review Organizations 
Agency for Healthcare Research and Quality 
Campbell Collaboration (social welfare/education) 
Centre for Reviews and Dissemination 
Cochrane Collaboration (health/medicine) 
EPPI-Centre (social sciences/education) 
International Initiative for Impact Evaluation 
Joanna Briggs Institute 
and many more
Stages of Systematic Review Process 
Stage 1 
Problem Formulation 
Stage 2 
Data Collection 
Stage 3 
Data Evaluation 
Stage 4 
Data Analysis 
Stage 5 
Reporting Findings 
Campbell Collaboration
Stage 1: Problem Formulation 
Stage 2 
Data Collection 
Stage 3 Data Collection 
Stage 4 
Data Analysis 
Stage 5 
Reporting Findings 
Stage 1 Problem Formulation
Problem Formulation 
Developing your protocol (research plan) 
Developing research questions 
Developing inclusion / exclusion criteria 
Developing a coding manual for data extraction 
Determining data management
Research Questions 
Questions about intervention effects 
Are juvenile drug courts effective in reducing drug crime recidivism among adolescents, relative to practice as usual? 
Questions about associations 
What is the relationship between teacher education/training and student achievement? Does this relationship differ for elementary and secondary school students? 
Diagnostic/Prognostic questions 
Which risk assessment (A vs. B) is a better predictor of child maltreatment?
Research Questions 
Questions often use the PICO(S) framework: 
Population/Participants 
Interventions 
Comparisons 
Outcomes 
Study design 
“To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
Research Questions 
Questions often use the PICO(S) framework: 
Population/Participants 
Interventions 
Comparisons 
Outcomes 
Study design 
“To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
Research Questions 
In older adults(P), is an educational intervention(I) more effective than no intervention or an alternative intervention(C) in 
1.improving driving awareness/knowledge, 
2.improving on-road driving behavior and skills, and, 
3.reducing crash rates? (O)
Research Questions 
In post-cardiac arrest patients with return of spontaneous circulation (P), does therapeutic hypothermia (I) compared with usual care (C), improve morbidity or mortality (O)?
Research Questions 
Developing a research question for a systematic review requires an understanding of the existing literature, including gaps and uncertainties, definitions, and terminologies
Inclusion / Exclusion Criteria 
Study eligibility criteria: 
Clearly defines which subjects or studies will be included in the review 
Essential for determining relevant and irrelevant studies 
Determined by the research question 
MUST be defined before data collection (i.e. conducting the search)
Inclusion / Exclusion Criteria 
Criteria to consider: 
Population/ Participants: 
Age / grade level 
Gender, ethnicity, socioeconomic status 
Special or clinical populations 
Interventions 
What types of interventions are relevant 
Identify elements of a program/intervention that are critical 
Outcomes 
What is the purpose of the intervention (e.g. quality of life, transition, etc) 
Study Design 
What study designs are most appropriate (e.g. RCT)
Inclusion / Exclusion Criteria 
Criteria to consider: 
Geographic considerations 
Time restrictions 
If limiting the research to specific dates, indicate reason 
Example: interventions for low birth weight babies – date limit was 1992 because of surfactant trials 
Language restrictions 
Remember, it can be difficult to obtain articles in Portuguese even if you are fluent
Inclusion / Exclusion Criteria 
Objective: Investigate the effectiveness of peer-mediated interventions (PMI) in the treatment of individuals with autism spectrum disorders (ASD)
Inclusion / Exclusion Criteria 
Note: SPIO is similar to PICO 
Objective: To determine the effectiveness of mindfulness-based interventions with people diagnosed with multiple sclerosis
Inclusion / Exclusion Criteria
Inclusion / Exclusion Criteria 
Physical exercise and individuals with autism spectrum disorders (ASD): 
Objective: review of interventions designed to increase the exercise behavior of individuals with ASD 
Inclusion Criteria 
Exclusion Criteria
Coding Manual 
Data extraction 
A coding manual to extract data (i.e. study details) from each study needs to be developed prior to collecting data (i.e. searching) 
Will be discussed under Data Extraction section
Data Management 
PRISMA – Preferred Reporting Items for Systematic Reviews and Meta-Analyses. 
It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. 
The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews. 
The PRISMA Statement consists of a 27-item checklist and a four- phase flow diagram.
Data Management 
PRISMA 27-item checklist 
http://www.prisma-statement.org/statement.htm
Data Management
Data Management
Data Management
Data Management 
Bibliographic Management 
Determine at the beginning of the research planning, what bibliographic management software you will use to manage and track data (i.e. references, citations) 
Endnote ($) 
UofC Library supports 
Refworks (free) 
UofC Library supports 
Mendeley (free) 
UofC Library supports 
Zotero (free) 
Reference Manager ($)
Data Management 
Use bibliographic management software to: 
Maintain a searchable database of references related to the systematic review 
Store all references selected for the systematic review 
Keep track of number of references from each database 
Determine number of duplicate references 
Store all discarded references 
Create citations and bibliography when writing up the results of the SR
Stage 2: Data Collection 
Stage 1 Problem Formulation 
Stage 3 
Data Evaluation 
Stage 4 Data Analysis 
Stage 5 
Reporting Findings 
Stage 2 Data Collection
Steps in Data Collection 
The Literature Search 
Developing inclusion / exclusion criteria 
Identifying resources to search
The Literature Search 
The literature search is the most important part of a systematic review (Campbell Collaboration) 
The literature search is how you obtain your data for your study 
The goal of the literature search is to find all relevant studies including: 
Traditional peer reviewed literature 
Grey literature 
The literature search must be comprehensive and systematic
The Literature Search 
The systematic review literature search must: 
Be well documented 
Be transparent and reproducible 
Include a diversity of resources 
Be iterative
The Literature Search 
Consult with your subject liaison librarian 
Document Document Document 
Databases searched, date range searched, date last searched 
All search terms for each database, as far as possible 
Follow the PRISMA (or other) flowchart 
Number of hits for each database searched 
Duplicates within databases
The Literature Search 
Search Strategy 
Key concepts to be searched (PICOS if using) 
How are these terms represented in different disciplines? 
What are related terms / synonyms? Be as comprehensive as possible 
What are the subject headings (e.g. MeSH) in each database? 
Review known relevant articles to determine keywords and subject headings 
Review previous relevant systematic reviews for possible strategy 
Is there a Cochrane group that has recommended terms
The Literature Search 
Search Strategy 
Use truncation 
nurs* will retrieve: nursing, nurse, nurses, nurses, nursed, etc 
Use both free text (keywords) and subject headings (i.e. controlled vocabulary), when possible 
Pilot your search strategy 
Is your search retrieving the known articles? 
If no, why not? 
Check indexing 
Check truncation 
Check free text keywords 
Check descriptors / subject headings 
It takes a LOT OF TIME
The Literature Search 
SSRIs 
Autism 
Children 
Selective serotonin reuptake inhibitor* 
OR 
Serotonin update Inhibitor 
OR 
5-ht update inhibitor* 
OR 
SSRI 
Autis* 
OR 
Infantile autism 
OR 
Autistic disorder 
OR 
ASD 
OR 
Asperger* 
Child* 
OR 
Infan* 
OR 
Todder* 
OR 
Preschool* 
Are SSRIs effective for treating autism in children? 
AND 
AND
The Literature Search 
What are the effective pre-graduation interventions (programs, strategies, skill training, etc) aimed at persons with autism spectrum disorders that result in employment in mainstream competitive employment settings? 
Autism 
Age 
Intervention 
Employment 
AND 
AND 
AND
The Literature Search 
Save your searches in each database 
Create an account within the database interface 
Label the search clearly (is it a draft, final, date, database name) 
Keep detailed records of all searches conducted: 
Month, date, year of search 
Database searched 
Exact search strategy 
Any limits used (year of publication, language, etc) 
Number of hits for each database 
Remember – goal is to report search strategy in a transparent manner so that it can be replicated
The Literature Search 
PRESS (Peer Review of Electronic Search Strategies) 
Translation: Is the search question translated well into search concepts? 
Operators: Are there any mistakes in the use of Boolean or proximity operators (e.g. ADJ) 
Subject Headings: are any important subject headings missing or have irrelevant ones been included? 
Natural language: are any natural language (i.e. free text) terms or spelling variances missing? Is truncation used correctly?
The Literature Search 
PRESS (Peer Review of Electronic Search Strategies) 
Spelling & Syntax: does the search strategy have any spelling mistakes, system syntax errors, or wrong line numbers? 
Limits: Do any of the limits used seem unnecessary or inappropriately used? 
Adapted for database: has the search strategy been adapted for each database to be searched? 
It is a good idea to use PRESS with other researchers and your subject liaison librarian to review your search strategy systematically
Identifying Sources 
Data sources 
Databases 
Reference searching 
Cited bys (forward citations searching, citation pearl growing) 
Contact with experts 
Hand searching key journals, journals with poor/no indexing 
Ongoing studies 
Grey literature
Databases 
UofC Library databases: 
http://library.ucalgary.ca/search- collections/databases-subject-or-name 
Ask yourself: which disciplines inform my research 
Educational interventions for young adults with ASD for gaining employment 
Different interfaces require different searching techniques 
Review online tutorials 
Meet with your liaison librarian
Grey Literature 
Grey literature is defined as: 
publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
Grey Literature 
Grey literature is defined as: 
publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
Grey Literature 
Grey Literature Includes: 
Government reports 
Committee reports 
Academic papers 
Theses / dissertations 
Conference papers & abstracts 
Conference proceedings 
Newsletters 
PPT presentations 
Discussion reports 
Program evaluation reports 
Technical documents 
Standards / best practice 
Working papers 
. . . and more 
Alberani V., Pietrangeli P. & Mazza A. (1990). The use of grey literature in health sciences: A preliminary survey. 
Bulletin of the Medical Library Association, 78, 358–363.
Grey Literature 
Locating Grey Literature: 
Consider what types of organizations or associations are likely to be interested in the same research question: 
Government agencies / NGO agencies 
Government departments 
Industry, trade, professional organizations 
Advocacy groups 
Private agencies
Grey Literature 
Locating Grey Literature: 
Canadian 
Canadian Health Research Collection 
Canadian Public Policy Collection 
OpenGrey 
Resource of grey literature in Europe 
Grey Literature Report 
New York Academy of Medicine 
Dissertations / Theses 
Proquest Dissertations and Theses 
Refer to the Systematic Review libguide for more grey literature resource
The Literature Search 
Interprofessional simulation-based education (IPSE) is becoming an increasingly popular educational strategy worldwide within undergraduate healthcare curricular. The purpose of the literature review was to examine research studies featuring undergraduate IPSE.
The Literature Search
The Literature Search 
The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer- reviewed literature.
The Literature Search 
We therefore conducted a systematic review to determine the comparative efficacy of NPWT versus alternate TAC techniques on in-hospital mortality, length of hospital or intensive care unit (ICU) stay, fascial closure rate, and adverse events in critically ill adults with open abdominal wounds.
PRISMA Flowchart
Stage 3: Data Evaluation 
Stage 1 
Problem Formulation 
Stage 2 
Data Collection 
Stage 4 Data Analysis 
Stage 5 
Reporting Findings 
Stage 3 Data Evaluation
Data Coding 
Data extraction is the process by which researchers obtain the necessary information about study characteristics and findings 
Standardized data extraction provides consistency, thereby potentially reducing bias, improving validity and reliability 
Ensures that the same information is obtained (extracted) from each study 
Should be developed early in the project 
Pilot the data extraction form with a sample of studies 
Keep track of revisions, corrections, amendments 
If possible, pilot the exporting, analysis, and outputs of the data extraction form
Data Coding 
Purpose: 
Provides an accounting of the research included in your review 
Identifies key components of the research including interventions, subjects, and methods
Data Coding 
Levels of Coding 
Abstract screening 
Study eligibility screening 
Study content coding
Abstract Data Coding 
Abstract Screening 
Is the study relevant based on the title and abstract 
Apply title/ abstract eligibility criteria, which are often less specific than the full inclusion/exclusion criteria 
If yes, retrieve full text 
If unclear, retrieve full text 
Exclude obviously irrelevant studies 
Include two trained raters working independently if possible
Full-Text Data Coding 
Full-Text Eligibility Screening 
Develop an eligibility screening form based on the inclusion / exclusion criteria 
Complete for each study retrieved that is potentially eligible 
Include reasons for ineligible studies (for PRISMA reporting)
Study Content Coding 
Data extraction form: 
Be consistent in the order and style you use to describe the information from each study 
Provide instructions, decision rules, and training on the use of the data extraction form 
Practice using the data extraction form 
Compare completed pilot forms 
Ensure that there is consistency in extracting data 
Ideally two researchers should independently perform data extraction 
Record any missing information as unclear or not described 
Makes it clear that the information was not included in the study rather than you forgot to extract it
Study Content Coding 
Systemtaic reviews: CRD’s guidance for undertaking systematic reviews in health care. 2009. Retrieved from: https://www.york.ac.uk/inst/crd/index_guidance.htm
Study Content Coding 
Cochrane has developed templates that can be revised 
Google: “data extraction form” cochrane 
Review published systematic reviews relevant to your research area 
Request permission to adapt the data extraction form
Study Content Coding 
Software for data extraction 
RevMan (Cochrane – can use free of charge for academic purposes) 
EPPI-Reviewer (one month free trial) 
JBI Sumari (subscription based) 
NVivo (UofC site license) 
Abstraktr 
Excel
Stage 4: Data Analysis 
Stage 1 
Problem Formulation 
Stage 2 
Data Collection 
Stage 3 
Data Evaluation 
Stage 5 
Reporting Findings 
Stage 4 Data Analysis
Data Analysis 
Planned analysis of included studies: 
Meta-analysis and/or narrative analysis
Data Analysis 
Analyzing the Evidence 
Each included study is analyzed for bias and quality 
Can be component of data extraction form 
Cochrane risk of bias framework 
GRADE system 
Method quality / critical appraisal checklists 
Quantitative 
Qualitative 
Critical appraisal resources: 
http://fhswedge.csu.mcmaster.ca/cepftp/qasite/CriticalAppraisal.html
Stage 5: Reporting Findings 
Stage 1 
Problem Formulation 
Stage 2 
Data Collection 
Stage 3 
Data Evaluation 
Stage 4 
Data Analysis 
Stage 5 Reporting Findings
Reporting Findings
Reporting Findings 
If you plan on publishing in a journal 
Review the intended journal for specific guidelines for systematic reviews BEFORE starting your review 
If there are no guidelines, review other systematic reviews in the intended journal
Checklists 
Refer to the PRISMA checklist 
How does your systematic review stand up? Review: Appraisal of a systematic review using a checklist: Notes and example 1 Clinical Evidence (BMJ) http://clinicalevidence.bmj.com/x/set/static/ebm/toolbox/665052.html
Alix Hayden KNES, NURS, EDUC, CORE ahayden@ucalgary.ca 
Laura Koltutsky PSYC, SOWK, Sociology 
ljkoltut@ucalgary.ca 
Helen Lee Robertson Medicine helen.lee.robertson@ucalgary.ca 
Contact:

More Related Content

What's hot

Basics of Systematic Review and Meta-analysis: Part 3
Basics of Systematic Review and Meta-analysis: Part 3Basics of Systematic Review and Meta-analysis: Part 3
Basics of Systematic Review and Meta-analysis: Part 3Rizwan S A
 
Introduction to systematic reviews
Introduction to systematic reviewsIntroduction to systematic reviews
Introduction to systematic reviewsOmar Midani
 
Systematic Reviews in the Health Sciences
Systematic Reviews in the Health SciencesSystematic Reviews in the Health Sciences
Systematic Reviews in the Health SciencesBecky Morin
 
Introduction to Systematic Review & Meta-Analysis
Introduction to Systematic Review & Meta-Analysis Introduction to Systematic Review & Meta-Analysis
Introduction to Systematic Review & Meta-Analysis Hasanain Ghazi
 
Meta analysis ppt
Meta analysis pptMeta analysis ppt
Meta analysis pptSKVA
 
Basics of Systematic Review and Meta-analysis: Part 2
Basics of Systematic Review and Meta-analysis: Part 2Basics of Systematic Review and Meta-analysis: Part 2
Basics of Systematic Review and Meta-analysis: Part 2Rizwan S A
 
Systematic review ppt
Systematic review pptSystematic review ppt
Systematic review pptBasil Asay
 
Systematic review and meta analaysis course - part 2
Systematic review and meta analaysis course - part 2Systematic review and meta analaysis course - part 2
Systematic review and meta analaysis course - part 2Ahmed Negida
 
Systematic reviews
Systematic reviewsSystematic reviews
Systematic reviewsZera Day
 
演講-Meta analysis in medical research-張偉豪
演講-Meta analysis in medical research-張偉豪演講-Meta analysis in medical research-張偉豪
演講-Meta analysis in medical research-張偉豪Beckett Hsieh
 
Meta analysis
Meta analysisMeta analysis
Meta analysisJunaidAKG
 
Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1Rizwan S A
 
Overview of systematic review and meta analysis
Overview of systematic review and meta  analysisOverview of systematic review and meta  analysis
Overview of systematic review and meta analysisDrsnehas2
 
General Introduction to Health research (Basic)
General Introduction to Health research (Basic)General Introduction to Health research (Basic)
General Introduction to Health research (Basic)Public Health Update
 
How to Conduct a Systematic Search in PubMed
How to Conduct a Systematic Search in PubMedHow to Conduct a Systematic Search in PubMed
How to Conduct a Systematic Search in PubMedRobin Featherstone
 

What's hot (20)

Basics of Systematic Review and Meta-analysis: Part 3
Basics of Systematic Review and Meta-analysis: Part 3Basics of Systematic Review and Meta-analysis: Part 3
Basics of Systematic Review and Meta-analysis: Part 3
 
Introduction to systematic reviews
Introduction to systematic reviewsIntroduction to systematic reviews
Introduction to systematic reviews
 
Systematic review
Systematic reviewSystematic review
Systematic review
 
Systematic Reviews in the Health Sciences
Systematic Reviews in the Health SciencesSystematic Reviews in the Health Sciences
Systematic Reviews in the Health Sciences
 
Introduction to Systematic Review & Meta-Analysis
Introduction to Systematic Review & Meta-Analysis Introduction to Systematic Review & Meta-Analysis
Introduction to Systematic Review & Meta-Analysis
 
Systematic Review & Meta-Analysis Course - Summary Slides
Systematic Review & Meta-Analysis Course - Summary SlidesSystematic Review & Meta-Analysis Course - Summary Slides
Systematic Review & Meta-Analysis Course - Summary Slides
 
Meta analysis ppt
Meta analysis pptMeta analysis ppt
Meta analysis ppt
 
Basics of Systematic Review and Meta-analysis: Part 2
Basics of Systematic Review and Meta-analysis: Part 2Basics of Systematic Review and Meta-analysis: Part 2
Basics of Systematic Review and Meta-analysis: Part 2
 
Overview of Evidence Based Medicine and Systematic Review Methodology
Overview of Evidence Based Medicine and Systematic Review MethodologyOverview of Evidence Based Medicine and Systematic Review Methodology
Overview of Evidence Based Medicine and Systematic Review Methodology
 
Systematic review ppt
Systematic review pptSystematic review ppt
Systematic review ppt
 
Systematic review and meta analaysis course - part 2
Systematic review and meta analaysis course - part 2Systematic review and meta analaysis course - part 2
Systematic review and meta analaysis course - part 2
 
Systematic reviews
Systematic reviewsSystematic reviews
Systematic reviews
 
演講-Meta analysis in medical research-張偉豪
演講-Meta analysis in medical research-張偉豪演講-Meta analysis in medical research-張偉豪
演講-Meta analysis in medical research-張偉豪
 
Meta analysis
Meta analysisMeta analysis
Meta analysis
 
Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1
 
Overview of systematic review and meta analysis
Overview of systematic review and meta  analysisOverview of systematic review and meta  analysis
Overview of systematic review and meta analysis
 
General Introduction to Health research (Basic)
General Introduction to Health research (Basic)General Introduction to Health research (Basic)
General Introduction to Health research (Basic)
 
How to Conduct a Systematic Search in PubMed
How to Conduct a Systematic Search in PubMedHow to Conduct a Systematic Search in PubMed
How to Conduct a Systematic Search in PubMed
 
Health research methodology
Health research methodologyHealth research methodology
Health research methodology
 
PICO Research Question
PICO Research QuestionPICO Research Question
PICO Research Question
 

Similar to Introduction to Systematic Reviews

review of Litt on clinical research and Methodology
review of Litt on clinical research and Methodologyreview of Litt on clinical research and Methodology
review of Litt on clinical research and MethodologyDr.Venkata Suresh Ponnuru
 
مراجعة الأدبيات المنهجيةsystematic literature review .ppt
مراجعة الأدبيات المنهجيةsystematic literature review .pptمراجعة الأدبيات المنهجيةsystematic literature review .ppt
مراجعة الأدبيات المنهجيةsystematic literature review .ppt901202
 
Developing a Systematic Review Protocol
Developing a Systematic Review ProtocolDeveloping a Systematic Review Protocol
Developing a Systematic Review ProtocolACSRM
 
How to conduct a systematic review
How to conduct a systematic reviewHow to conduct a systematic review
How to conduct a systematic reviewDrNidhiPruthiShukla
 
Research Overview
Research OverviewResearch Overview
Research OverviewS A Tabish
 
evidence based periodontics
 evidence based periodontics    evidence based periodontics
evidence based periodontics neeti shinde
 
Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...
Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...
Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...NHSNWRD
 
PHAA_SRworkshop07.ppt
PHAA_SRworkshop07.pptPHAA_SRworkshop07.ppt
PHAA_SRworkshop07.pptssuser50a5ec
 
Systematic Review-Scoping Review
Systematic Review-Scoping ReviewSystematic Review-Scoping Review
Systematic Review-Scoping ReviewNeda Firouraghi
 
A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...
A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...
A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...Janelle Martinez
 
Systematic Reviews Overview
Systematic Reviews OverviewSystematic Reviews Overview
Systematic Reviews Overviewralloyd79
 
Edmedia2011 lit.sys.review
Edmedia2011 lit.sys.reviewEdmedia2011 lit.sys.review
Edmedia2011 lit.sys.reviewValerie Irvine
 
Systematic and Scoping Reviews.pptx
Systematic and Scoping Reviews.pptxSystematic and Scoping Reviews.pptx
Systematic and Scoping Reviews.pptxLauraLarreaMantilla
 
Living evidence 3
Living evidence 3Living evidence 3
Living evidence 3stanbridge
 
Introa-Morton-slides.pdf
Introa-Morton-slides.pdfIntroa-Morton-slides.pdf
Introa-Morton-slides.pdfShahriarHabib4
 

Similar to Introduction to Systematic Reviews (20)

review of Litt on clinical research and Methodology
review of Litt on clinical research and Methodologyreview of Litt on clinical research and Methodology
review of Litt on clinical research and Methodology
 
مراجعة الأدبيات المنهجيةsystematic literature review .ppt
مراجعة الأدبيات المنهجيةsystematic literature review .pptمراجعة الأدبيات المنهجيةsystematic literature review .ppt
مراجعة الأدبيات المنهجيةsystematic literature review .ppt
 
MA Talk Temple.ppt
MA Talk Temple.pptMA Talk Temple.ppt
MA Talk Temple.ppt
 
Developing a Systematic Review Protocol
Developing a Systematic Review ProtocolDeveloping a Systematic Review Protocol
Developing a Systematic Review Protocol
 
How to conduct a systematic review
How to conduct a systematic reviewHow to conduct a systematic review
How to conduct a systematic review
 
Research Overview
Research OverviewResearch Overview
Research Overview
 
evidence based periodontics
 evidence based periodontics    evidence based periodontics
evidence based periodontics
 
Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...
Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...
Let's Talk Research Annual Conference - 24th-25th September 2014 (Professor R...
 
PHAA_SRworkshop07.ppt
PHAA_SRworkshop07.pptPHAA_SRworkshop07.ppt
PHAA_SRworkshop07.ppt
 
Systematic Review-Scoping Review
Systematic Review-Scoping ReviewSystematic Review-Scoping Review
Systematic Review-Scoping Review
 
What Is Evidence Based Practice
What Is Evidence Based Practice What Is Evidence Based Practice
What Is Evidence Based Practice
 
A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...
A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...
A Practical Guide To Writing And Reviewing Abstracts For Pediatric Academic M...
 
Systematic Reviews Overview
Systematic Reviews OverviewSystematic Reviews Overview
Systematic Reviews Overview
 
Edmedia2011 lit.sys.review
Edmedia2011 lit.sys.reviewEdmedia2011 lit.sys.review
Edmedia2011 lit.sys.review
 
Systematic and Scoping Reviews.pptx
Systematic and Scoping Reviews.pptxSystematic and Scoping Reviews.pptx
Systematic and Scoping Reviews.pptx
 
Action research
Action researchAction research
Action research
 
Study Eligibility Criteria
Study Eligibility CriteriaStudy Eligibility Criteria
Study Eligibility Criteria
 
Living evidence 3
Living evidence 3Living evidence 3
Living evidence 3
 
Introa-Morton-slides.pdf
Introa-Morton-slides.pdfIntroa-Morton-slides.pdf
Introa-Morton-slides.pdf
 
Methodology
MethodologyMethodology
Methodology
 

Recently uploaded

How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....Riddhi Kevadiya
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...M56BOOKSTORE PRODUCT/SERVICE
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptxmary850239
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
EBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting BlEBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting BlDr. Bruce A. Johnson
 
SOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptx
SOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptxSOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptx
SOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptxSyedNadeemGillANi
 
Optical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxOptical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxPurva Nikam
 
How to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using CodeHow to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using CodeCeline George
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
10 Topics For MBA Project Report [HR].pdf
10 Topics For MBA Project Report [HR].pdf10 Topics For MBA Project Report [HR].pdf
10 Topics For MBA Project Report [HR].pdfJayanti Pande
 
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...Dr. Asif Anas
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17Celine George
 

Recently uploaded (20)

How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
EBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting BlEBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting Bl
 
SOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptx
SOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptxSOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptx
SOLIDE WASTE in Cameroon,,,,,,,,,,,,,,,,,,,,,,,,,,,.pptx
 
Finals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quizFinals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quiz
 
Optical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxOptical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptx
 
How to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using CodeHow to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using Code
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
10 Topics For MBA Project Report [HR].pdf
10 Topics For MBA Project Report [HR].pdf10 Topics For MBA Project Report [HR].pdf
10 Topics For MBA Project Report [HR].pdf
 
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17
 

Introduction to Systematic Reviews

  • 1. INTRODUCTION TO THE SYSTEMATIC REVIEW PROCESS Libraries and Cultural Resources Alix Hayden, Laura Koltutsky, Helen Lee Robertson May 7, 2014
  • 2. Guide Content The content in this guide is adapted from: Tanner, E. E. & Wilson S. J. (2013) Systematic Reviewing for Evidence-Based Practice: An Introductory Workshop. Systematic Reviews: CRD’s guidance for undertaking systematic reviews in health care. Personal expertise, experience, and knowledge of the presenters.
  • 3. Systematic Review Process Centre for Health Communication and Participation http://navigatingeffectivetreatments.org.au/exploring_systematic_reviews.html
  • 5. Pyramid of Evidence In Vitro Research Animal Research Ideas, Editorials, Opinions Case Reports, Case Series Case-Controlled Studies Prospective Cohort Studies Quasi-Experimental Randomized Controlled Double Blind Studies Systematic Reviews, Meta-Analyses
  • 8. Systematic Review Definitions This methodology prescribes explicit, reproducible, and transparent processes for collating the best available evidence in answer to specific questions. In particular, it requires the use of robust techniques for: searching for and identifying primary studies, selecting the studies to be included in the review, extracting the data from the studies, and appraising the quality of these studies, synthesizing the findings narratively and/or through pooling suitable quantitative data in meta-analysis Dixon-Woods, M., & Sutton, A. (2004). Systematic Review. In Michael S. Lewis-Beck, A. Bryman, & Tim Futing Liao (Eds.), The SAGE Encyclopedia of Social Science Research Methods. (pp. 1111-1112). Thousand Oaks, CA: Sage Publications, Inc. doi: http://dx.doi.org.ezproxy.lib.ucalgary.ca/10.4135/9781412950589.n998
  • 9. Systematic Reviews / Traditional Reviews Adapted from: Cook, D., Mulrow, C. & Haydens, R. (1997) Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5):376-380. Feature Systematic Review Narrative / Traditional Reviews Question Often focused (clinical) question. Sometimes broad in scope. Sources & Search Explicit search strategy of multiple databases. Comprehensive sources. Not usually specified. Selection Criterion-based selection; uniformly applied Not usually specified. Appraisal Rigorous critical appraisal. Variable. Synthesis Quantitative summary. Also qualitative/narrative. Often qualitative summary. Inferences Based on all available evidence. Based on a sample of the evidence. Grading All evidence is graded (quality) May or may not be graded.
  • 10. Systematic Review Myths Systematic reviews: are the same as ordinary literature reviews, only bigger include only randomized controlled trials require the adoption of a biomedical model of health are of no relevance outside of health / medicine must involve statistical analysis / synthesis must be conducted by experts can be done without experienced librarian support are not really research Adapted from: Petticrew, M. (2001) Systematic reviews from astronomy to zoology: Myths and misconceptions. BMJ 322:98-101.
  • 11. Reducing Bias Systematic reviews attempt to minimize bias and error throughout the review process Uses systematic process that is transparent and replicable Document transparent / replicable search strategies Inter-rater reliability on key data Assessment of bias in studies Assessment of study quality Meta-analyis (when possible) to statistically synthesize results across studies
  • 12. Why Conduct a Systematic Review Rigorous methods to summarize evidence on a given topic
  • 13. Timeline (Cochrane) Month Activity 1 – 2 Preparation of protocol 3 – 8 Searches for published and unpublished studies 2 – 3 Pilot test of eligibility criteria 3 – 8 Inclusion assessments 3 Pilot test “Risk of Bias” assessment 3 – 10 Validity assessments 3 Pilot test of data collection 3 – 10 Data collection 3 – 10 Data entry 5 – 11 Follow up of missing information 8 – 10 Analysis 1 – 11 Preparation of review report 12 - Keeping the review up-to-date
  • 14. Systematic Review Organizations Agency for Healthcare Research and Quality Campbell Collaboration (social welfare/education) Centre for Reviews and Dissemination Cochrane Collaboration (health/medicine) EPPI-Centre (social sciences/education) International Initiative for Impact Evaluation Joanna Briggs Institute and many more
  • 15. Stages of Systematic Review Process Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings Campbell Collaboration
  • 16. Stage 1: Problem Formulation Stage 2 Data Collection Stage 3 Data Collection Stage 4 Data Analysis Stage 5 Reporting Findings Stage 1 Problem Formulation
  • 17. Problem Formulation Developing your protocol (research plan) Developing research questions Developing inclusion / exclusion criteria Developing a coding manual for data extraction Determining data management
  • 18. Research Questions Questions about intervention effects Are juvenile drug courts effective in reducing drug crime recidivism among adolescents, relative to practice as usual? Questions about associations What is the relationship between teacher education/training and student achievement? Does this relationship differ for elementary and secondary school students? Diagnostic/Prognostic questions Which risk assessment (A vs. B) is a better predictor of child maltreatment?
  • 19. Research Questions Questions often use the PICO(S) framework: Population/Participants Interventions Comparisons Outcomes Study design “To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
  • 20. Research Questions Questions often use the PICO(S) framework: Population/Participants Interventions Comparisons Outcomes Study design “To assess the effects of [intervention] compared to [comparison/control] for [condition/problem] in [population] in [context] on [outcomes].”
  • 21. Research Questions In older adults(P), is an educational intervention(I) more effective than no intervention or an alternative intervention(C) in 1.improving driving awareness/knowledge, 2.improving on-road driving behavior and skills, and, 3.reducing crash rates? (O)
  • 22. Research Questions In post-cardiac arrest patients with return of spontaneous circulation (P), does therapeutic hypothermia (I) compared with usual care (C), improve morbidity or mortality (O)?
  • 23. Research Questions Developing a research question for a systematic review requires an understanding of the existing literature, including gaps and uncertainties, definitions, and terminologies
  • 24. Inclusion / Exclusion Criteria Study eligibility criteria: Clearly defines which subjects or studies will be included in the review Essential for determining relevant and irrelevant studies Determined by the research question MUST be defined before data collection (i.e. conducting the search)
  • 25. Inclusion / Exclusion Criteria Criteria to consider: Population/ Participants: Age / grade level Gender, ethnicity, socioeconomic status Special or clinical populations Interventions What types of interventions are relevant Identify elements of a program/intervention that are critical Outcomes What is the purpose of the intervention (e.g. quality of life, transition, etc) Study Design What study designs are most appropriate (e.g. RCT)
  • 26. Inclusion / Exclusion Criteria Criteria to consider: Geographic considerations Time restrictions If limiting the research to specific dates, indicate reason Example: interventions for low birth weight babies – date limit was 1992 because of surfactant trials Language restrictions Remember, it can be difficult to obtain articles in Portuguese even if you are fluent
  • 27. Inclusion / Exclusion Criteria Objective: Investigate the effectiveness of peer-mediated interventions (PMI) in the treatment of individuals with autism spectrum disorders (ASD)
  • 28. Inclusion / Exclusion Criteria Note: SPIO is similar to PICO Objective: To determine the effectiveness of mindfulness-based interventions with people diagnosed with multiple sclerosis
  • 30. Inclusion / Exclusion Criteria Physical exercise and individuals with autism spectrum disorders (ASD): Objective: review of interventions designed to increase the exercise behavior of individuals with ASD Inclusion Criteria Exclusion Criteria
  • 31. Coding Manual Data extraction A coding manual to extract data (i.e. study details) from each study needs to be developed prior to collecting data (i.e. searching) Will be discussed under Data Extraction section
  • 32. Data Management PRISMA – Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews. The PRISMA Statement consists of a 27-item checklist and a four- phase flow diagram.
  • 33. Data Management PRISMA 27-item checklist http://www.prisma-statement.org/statement.htm
  • 37. Data Management Bibliographic Management Determine at the beginning of the research planning, what bibliographic management software you will use to manage and track data (i.e. references, citations) Endnote ($) UofC Library supports Refworks (free) UofC Library supports Mendeley (free) UofC Library supports Zotero (free) Reference Manager ($)
  • 38. Data Management Use bibliographic management software to: Maintain a searchable database of references related to the systematic review Store all references selected for the systematic review Keep track of number of references from each database Determine number of duplicate references Store all discarded references Create citations and bibliography when writing up the results of the SR
  • 39. Stage 2: Data Collection Stage 1 Problem Formulation Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings Stage 2 Data Collection
  • 40. Steps in Data Collection The Literature Search Developing inclusion / exclusion criteria Identifying resources to search
  • 41. The Literature Search The literature search is the most important part of a systematic review (Campbell Collaboration) The literature search is how you obtain your data for your study The goal of the literature search is to find all relevant studies including: Traditional peer reviewed literature Grey literature The literature search must be comprehensive and systematic
  • 42. The Literature Search The systematic review literature search must: Be well documented Be transparent and reproducible Include a diversity of resources Be iterative
  • 43. The Literature Search Consult with your subject liaison librarian Document Document Document Databases searched, date range searched, date last searched All search terms for each database, as far as possible Follow the PRISMA (or other) flowchart Number of hits for each database searched Duplicates within databases
  • 44. The Literature Search Search Strategy Key concepts to be searched (PICOS if using) How are these terms represented in different disciplines? What are related terms / synonyms? Be as comprehensive as possible What are the subject headings (e.g. MeSH) in each database? Review known relevant articles to determine keywords and subject headings Review previous relevant systematic reviews for possible strategy Is there a Cochrane group that has recommended terms
  • 45. The Literature Search Search Strategy Use truncation nurs* will retrieve: nursing, nurse, nurses, nurses, nursed, etc Use both free text (keywords) and subject headings (i.e. controlled vocabulary), when possible Pilot your search strategy Is your search retrieving the known articles? If no, why not? Check indexing Check truncation Check free text keywords Check descriptors / subject headings It takes a LOT OF TIME
  • 46. The Literature Search SSRIs Autism Children Selective serotonin reuptake inhibitor* OR Serotonin update Inhibitor OR 5-ht update inhibitor* OR SSRI Autis* OR Infantile autism OR Autistic disorder OR ASD OR Asperger* Child* OR Infan* OR Todder* OR Preschool* Are SSRIs effective for treating autism in children? AND AND
  • 47. The Literature Search What are the effective pre-graduation interventions (programs, strategies, skill training, etc) aimed at persons with autism spectrum disorders that result in employment in mainstream competitive employment settings? Autism Age Intervention Employment AND AND AND
  • 48. The Literature Search Save your searches in each database Create an account within the database interface Label the search clearly (is it a draft, final, date, database name) Keep detailed records of all searches conducted: Month, date, year of search Database searched Exact search strategy Any limits used (year of publication, language, etc) Number of hits for each database Remember – goal is to report search strategy in a transparent manner so that it can be replicated
  • 49. The Literature Search PRESS (Peer Review of Electronic Search Strategies) Translation: Is the search question translated well into search concepts? Operators: Are there any mistakes in the use of Boolean or proximity operators (e.g. ADJ) Subject Headings: are any important subject headings missing or have irrelevant ones been included? Natural language: are any natural language (i.e. free text) terms or spelling variances missing? Is truncation used correctly?
  • 50. The Literature Search PRESS (Peer Review of Electronic Search Strategies) Spelling & Syntax: does the search strategy have any spelling mistakes, system syntax errors, or wrong line numbers? Limits: Do any of the limits used seem unnecessary or inappropriately used? Adapted for database: has the search strategy been adapted for each database to be searched? It is a good idea to use PRESS with other researchers and your subject liaison librarian to review your search strategy systematically
  • 51. Identifying Sources Data sources Databases Reference searching Cited bys (forward citations searching, citation pearl growing) Contact with experts Hand searching key journals, journals with poor/no indexing Ongoing studies Grey literature
  • 52. Databases UofC Library databases: http://library.ucalgary.ca/search- collections/databases-subject-or-name Ask yourself: which disciplines inform my research Educational interventions for young adults with ASD for gaining employment Different interfaces require different searching techniques Review online tutorials Meet with your liaison librarian
  • 53. Grey Literature Grey literature is defined as: publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
  • 54. Grey Literature Grey literature is defined as: publicly available, foreign or domestic, open source information that is usually available only through special channels and may not enter normal channels or systems of publication, distribution, bibliographic control, or acquisition by book sellers or subscription agents (U.S. Interagency Grey Literature Working Group 1995).
  • 55. Grey Literature Grey Literature Includes: Government reports Committee reports Academic papers Theses / dissertations Conference papers & abstracts Conference proceedings Newsletters PPT presentations Discussion reports Program evaluation reports Technical documents Standards / best practice Working papers . . . and more Alberani V., Pietrangeli P. & Mazza A. (1990). The use of grey literature in health sciences: A preliminary survey. Bulletin of the Medical Library Association, 78, 358–363.
  • 56. Grey Literature Locating Grey Literature: Consider what types of organizations or associations are likely to be interested in the same research question: Government agencies / NGO agencies Government departments Industry, trade, professional organizations Advocacy groups Private agencies
  • 57. Grey Literature Locating Grey Literature: Canadian Canadian Health Research Collection Canadian Public Policy Collection OpenGrey Resource of grey literature in Europe Grey Literature Report New York Academy of Medicine Dissertations / Theses Proquest Dissertations and Theses Refer to the Systematic Review libguide for more grey literature resource
  • 58. The Literature Search Interprofessional simulation-based education (IPSE) is becoming an increasingly popular educational strategy worldwide within undergraduate healthcare curricular. The purpose of the literature review was to examine research studies featuring undergraduate IPSE.
  • 60. The Literature Search The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer- reviewed literature.
  • 61. The Literature Search We therefore conducted a systematic review to determine the comparative efficacy of NPWT versus alternate TAC techniques on in-hospital mortality, length of hospital or intensive care unit (ICU) stay, fascial closure rate, and adverse events in critically ill adults with open abdominal wounds.
  • 63. Stage 3: Data Evaluation Stage 1 Problem Formulation Stage 2 Data Collection Stage 4 Data Analysis Stage 5 Reporting Findings Stage 3 Data Evaluation
  • 64. Data Coding Data extraction is the process by which researchers obtain the necessary information about study characteristics and findings Standardized data extraction provides consistency, thereby potentially reducing bias, improving validity and reliability Ensures that the same information is obtained (extracted) from each study Should be developed early in the project Pilot the data extraction form with a sample of studies Keep track of revisions, corrections, amendments If possible, pilot the exporting, analysis, and outputs of the data extraction form
  • 65. Data Coding Purpose: Provides an accounting of the research included in your review Identifies key components of the research including interventions, subjects, and methods
  • 66. Data Coding Levels of Coding Abstract screening Study eligibility screening Study content coding
  • 67. Abstract Data Coding Abstract Screening Is the study relevant based on the title and abstract Apply title/ abstract eligibility criteria, which are often less specific than the full inclusion/exclusion criteria If yes, retrieve full text If unclear, retrieve full text Exclude obviously irrelevant studies Include two trained raters working independently if possible
  • 68. Full-Text Data Coding Full-Text Eligibility Screening Develop an eligibility screening form based on the inclusion / exclusion criteria Complete for each study retrieved that is potentially eligible Include reasons for ineligible studies (for PRISMA reporting)
  • 69. Study Content Coding Data extraction form: Be consistent in the order and style you use to describe the information from each study Provide instructions, decision rules, and training on the use of the data extraction form Practice using the data extraction form Compare completed pilot forms Ensure that there is consistency in extracting data Ideally two researchers should independently perform data extraction Record any missing information as unclear or not described Makes it clear that the information was not included in the study rather than you forgot to extract it
  • 70. Study Content Coding Systemtaic reviews: CRD’s guidance for undertaking systematic reviews in health care. 2009. Retrieved from: https://www.york.ac.uk/inst/crd/index_guidance.htm
  • 71. Study Content Coding Cochrane has developed templates that can be revised Google: “data extraction form” cochrane Review published systematic reviews relevant to your research area Request permission to adapt the data extraction form
  • 72. Study Content Coding Software for data extraction RevMan (Cochrane – can use free of charge for academic purposes) EPPI-Reviewer (one month free trial) JBI Sumari (subscription based) NVivo (UofC site license) Abstraktr Excel
  • 73. Stage 4: Data Analysis Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 5 Reporting Findings Stage 4 Data Analysis
  • 74. Data Analysis Planned analysis of included studies: Meta-analysis and/or narrative analysis
  • 75. Data Analysis Analyzing the Evidence Each included study is analyzed for bias and quality Can be component of data extraction form Cochrane risk of bias framework GRADE system Method quality / critical appraisal checklists Quantitative Qualitative Critical appraisal resources: http://fhswedge.csu.mcmaster.ca/cepftp/qasite/CriticalAppraisal.html
  • 76. Stage 5: Reporting Findings Stage 1 Problem Formulation Stage 2 Data Collection Stage 3 Data Evaluation Stage 4 Data Analysis Stage 5 Reporting Findings
  • 78. Reporting Findings If you plan on publishing in a journal Review the intended journal for specific guidelines for systematic reviews BEFORE starting your review If there are no guidelines, review other systematic reviews in the intended journal
  • 79. Checklists Refer to the PRISMA checklist How does your systematic review stand up? Review: Appraisal of a systematic review using a checklist: Notes and example 1 Clinical Evidence (BMJ) http://clinicalevidence.bmj.com/x/set/static/ebm/toolbox/665052.html
  • 80. Alix Hayden KNES, NURS, EDUC, CORE ahayden@ucalgary.ca Laura Koltutsky PSYC, SOWK, Sociology ljkoltut@ucalgary.ca Helen Lee Robertson Medicine helen.lee.robertson@ucalgary.ca Contact: