This document provides an overview of systematic literature reviews. It defines systematic reviews as reviews that use explicit and reproducible methods to identify, select, and critically appraise relevant research to answer a specific question. The key steps outlined include developing a protocol, formulating a review question using PICO elements, establishing inclusion/exclusion criteria, systematically searching literature sources, selecting studies, assessing study quality, extracting data, synthesizing results, and interpreting findings. Examples are provided for many of the steps like developing search strategies, creating logs to document the process, and tools for summarizing evidence like PRISMA diagrams and data tables.

1. 1
Systematic (Literature) Review
Dr. Khalid Mahmood
Professor
Department of Information Management
University of the Punjab

2.  Professor of Information Management at University of the
Punjab
 Post-doctoral research fellow at University of California, Loss
Angeles, USA
 150+ publications
 Supervised many doctoral, M.Phil. and master theses
 Worked for various research journals as editor, reviewer and
editorial board member
 Conducted many trainings on research writing and publishing
About me
2

3. Acknowledgment
 I have prepared this presentation with the help of
many books, presentations and Websites.
 I pay my sincere gratitude to all authors,
professors and experts for their efforts and
contributions.
3

4. 4
Literature review
 The general term for all attempts to
synthesize the results and conclusions of
two or more publications on a given topic.
A review may or may not be systematic.

5.  Narrative review
 Traditional expert review; usually subjective in nature
 Systematic review
 A review of a clearly formulated question that uses systematic
and explicit methods to identify, select and critically appraise
relevant research, and to collect and analyze data from the
studies that are included in the review
 Meta-analysis
 Quantitative evidence
 Use of statistical methods to combine the results of various
independent, similar studies
 More precise calculation of one estimate of treatment effect than
could be achieved by any of the individual, contributing studies
5
Levels of literature review

6. Systematic review – history
1
1
1
272
334
371
323
386
429
482
596
639
741
849
948
1079
1289
1594
2063
2335
2596
2778
3239
3930
4850
6538
8113
7096
526
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
 Astronomers claim to be the
first users of this method
 Explosion of SRs in health
sciences in mid 1980s
 Term “systematic review” was
coined by health care
researchers
 SR became a significant tool
for “evidence-based medicine”
or “evidence-based practice”
6
 Growth of SRs
in health
sciences

7. SR – another definition
A systematic review “attempts
 to collate all empirical evidence that fits pre-
specified eligibility criteria
 in order to answer a specific research question.
 It uses explicit, systematic methods that are
selected with a view to minimizing bias,
 thus providing more reliable findings from
which conclusions can be drawn and decisions
made.”
7

8. SR vs. narrative review
8

9. Characteristics of an SR
 Clearly stated set of objectives with pre-defined
eligibility criteria for studies
 Explicit, reproducible methodology
 Systematic search that attempts to identify all
studies that would meet the eligibility criteria
 Assessment of the validity of the findings of the
included studies
 Systematic presentation, and synthesis, of the
characteristics and findings of the included
studies
9

10. Steps in an SR
 Build a review team
 Develop a protocol or plan
 Formulate review question
 Define inclusion and exclusion criteria
 Locate studies
 Select studies
 Assess study quality
 Extract data
 Analysis/summary and synthesis of relevant studies
 Present results
 Interpret results/determining the applicability of results
10

11. Review team
 Normally a team work
 Key skills:
 Managing research projects
 Leading, coordinating
 Expertise in the topic
 Methodological expertise
 Planning, searching, managing information, coding,
analyzing, synthesizing, writing
11

12. Protocol
 What is the title?
 What is the context and what are the conceptual
issues?
 What is the aim?
 What is the research question?
 What is the search strategy?
 What are the inclusion / exclusion criteria?
 How will the data be extracted and analyzed?
 How will the quality of studies be assessed?
12

13. PICO – question components
in medicine
 P – Population
 Patients (Demographic factors, socioeconomic
factors, setting, etc.)
 I – Intervention
 Drug, procedure, etc.
 C – Comparison
 Alternative to compare with intervention (placebo or
active)
 O – Outcome
 Improvement, effect, measure, etc.
13

14. PICO – example
14

15. Sample questions
 In undergraduate medical education, does the
use of clicker technology in the classroom
improve learning outcomes?
 Are antiseptic washes more effective than non
antiseptic washes at preventing nosocomial
infections in patients undergoing surgery?
 Are mass media (or school-based or community-
based) interventions effective in preventing
smoking in young people?
15

16. Inclusion and exclusion
criteria – example
16

17. Searching literature
 Identifying major concepts
 Keywords, synonyms, controlled vocabulary
 Combination of concepts
 Boolean operators, string, truncation, proximity, etc.
 Identifying where to search
 Search strategy
 Varies in different databases
 Export citations to a reference management software
 EndNote, etc.
 Documenting your search
 Database name, date of searching, number of results
17

18. Sources of literature
 Electronic databases
 General vs. subject
 Grey literature
 Conference proceedings, theses, reports,
Websites
 Browsing issues of topical journals
 Backward and forward citations of the most
relevant articles
 Conversation with experts in the field
18

19. Search strategy – example
19

20. Search strategy – example
20

21. Record keeping log – example
21

22. Study selection
 An initial assessment that occurs following the search
 It addresses the question “should the paper be
retrieved?”
 It is essential to use two assessors in both the selection
and critical appraisal processes to limit the risk of error
 Select only those studies that address the review
question and that match the inclusion criteria
 Scan titles and abstracts
 If uncertain? - Retrieve - scan full text
22

23. PRISMA flow diagram
23

24. PRISMA diagram – example
24

25. Study quality assessment
 Choose appropriate checklist
 related to study design
 It is better to use more reviewers
 Inter-reviewer reliability
25

26. Quality assessment criteria
26

27. Quality assessment results
27

28. Data extraction
 Think about what data you need to extract
from included studies to answer the
questions
 Pilot a draft data extraction form
28

29. Data extraction form
29

30. Summary table of evidence
30

31. Data synthesis
 Will results be pooled? How?
 How will differences between studies be
taken into account?
 Can subgroups of data be made?
 How will results be displayed?
31

32. Best of luck for your
research endeavors!
32