The slides show how to conduct systematic literature review (SLR) in any field of research. It is highly important that any SLR should ultimately highlight potential future directions and research gaps so that prospect researchers may focus on those particular areas.

1. SYSTEMATIC LITERATURE REVIEW

2. SYSTEMATIC LITERATURE REVIEW
K A S H I F H U S S A I N
RESEARCHER | DATA SCIENTIST | SOFTWARE ENGINEER
usitsoft@Hotmail.com
• PhD. researcher with background in data mining, machine learning,
and data science.
• An experienced software engineer with hands on experience from
application development to maintenance to support and
documentation.
• Well-versed with .Net Platform, RDBMS, MATLAB, Python, R,
machine learning algorithms, and data visualization.
• Looking forward to applying development experience and research
proficiencies in real-life data science problems including big data.

3. SYSTEMATIC LITERATURE REVIEW
TO LEARN TODAY
 Why to do SLR
 When to do SLR
 How to do SLR

4. SYSTEMATIC LITERATURE REVIEW
WHY TO DO SLR?
1
Idea
Discovery
Conduct
Research
Findings
Publish
Findings
here we
need SLRs
Research
Community
Publishers

5. SYSTEMATIC LITERATURE REVIEW
WHY TO DO SLR?
2
https://www.elsevier.com/connect/elsevier-publishing-a-look-at-the-
numbers-and-more
Tom Reller
2015
2.5 Million articles submitted by
7.8 Million researchers

6. SYSTEMATIC LITERATURE REVIEW
WHY TO DO SLR?
3
https://www.elsevier.com/connect/scientific-controversies-systematic-review-can-help
Science is full of controversy!
 Studies produce conflicting results
 Multiple answers to same question
“With standardized methodologies for aggregating
and analyzing studies, you can make the best use of
the best evidence available. Ultimately, this
approach – called systematic review – saves time
and resources, avoids unnecessary research and, in
the case of medicine, saves lives.”

7. SYSTEMATIC LITERATURE REVIEW
WHY TO DO SLR?
4
Benefits for Community Benefits for Authors
• Summary of evidence
• Current status of research area
• Helps decide in case of scientific
conflicts
• Answer to several questions
• Potential research topics
• Increased readership
• Increased citation
• Increase h-index
“If I have seen a little farther than others, it is because I have
stood on the shoulders of giants.” Newton

8. SYSTEMATIC LITERATURE REVIEW
WHEN TO DO SLR?
5
• In the start of PhD or Master research
• When looking for new idea

9. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
6
• What is SLR?
• How to conduct?
Need to know

10. SYSTEMATIC LITERATURE REVIEW
WHAT IS SLR?
7
Budgen, D., & Brereton, P. (2006, May). Performing systematic literature reviews in software engineering.
In Proceedings of the 28th international conference on Software engineering (pp. 1051-1052). ACM.
“A form of secondary study that uses a well-defined
methodology to identify, analyse and interpret all
available evidence related to a specific research question
in a way that is unbiased and (to a degree) repeatable.”
Budgen, D., & Brereton, P. (2006, May).

11. SYSTEMATIC LITERATURE REVIEW
WHAT IS SLR?
8
1. Research Questions
2. Review Protocol
(Method & Research Questions)
3. Answers to Questions
SLR Features

12. SYSTEMATIC LITERATURE REVIEW
WHAT IS SLR?
9
SLR Approaches
1. Petersen et al. (2008)
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson,
M. (2008, June). Systematic Mapping Studies in
Software Engineering. In EASE (Vol. 8, pp. 68-77).
2. Keele (2007)
Keele S (2007) Guidelines for performing
systematic literature reviews in software
engineering. In: Technical report, Ver. 2.3 EBSE
Technical Report. EBSE. sn

13. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 0
Petersen, K. et al. (2008)
THE SYSTEMATIC MAPPING PROCESS
Definition of
Research
Question(s)
Review Scope
Conduct
Research
All Papers
Screening of
Papers
Relevant
Papers
Keywording
using Abstracts
Classification
Scheme
Data Extraction
and Mapping
Process
Systematic
Map
Process Steps
Outcome

14. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 1
Hussain, K. et al. (2018)
THE SYSTEMATIC MAPPING PROCESS
Preliminary
Study
Keywords,
Research
Questions,
Search Venues,
Review Scope
Conduct
Research
All Papers
Screening of
Papers
Relevant
Papers
Data Extraction
and Mapping
Process
Systematic
Map
Process Steps
Outcome

15. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 2
Hussain, K. et al. (2018)
THE SYSTEMATIC MAPPING PROCESS
Preliminary
Study
Keywords,
Research
Questions,
Search Venues,
Review Scope
Conduct
Research
All Papers
Screening of
Papers
Relevant
Papers
Data Extraction
and Mapping
Process
Systematic
Map
Process Steps
Outcome
Inclusion-Exclusion Criteria

16. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 3
Preliminary
Study
Keywords,
Research
Questions,
Search Venues,
Review Scope
Process Steps
Outcome
• Initially collect papers with broadly used terms
• Collect keywords from collected papers
• List down most common keywords
• First draft of questions, update throughout study
• Determine scope, update scope to a mature state

17. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 4
Preliminary
Study
Keywords,
Research
Questions,
Search Venues,
Review Scope
Process Steps
Outcome
• Initially collect papers with broadly used terms
• Review scope may be:
 Technique(s) (e.g. Hybrids of ANN and Fuzzy Logic)
 Method(s) (e.g. ANN training using PSO)
 Approach(s) (e.g. ANN training methods using metaheuristics)
 Region(s)
 Period of publication (e.g. previous decade)
 Specific factors (e.g. ANN applications in medicine)

18. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 5
Preliminary
Study
Keywords,
Research
Questions,
Search Venues,
Review Scope
Process Steps
Outcome
• Master student
 Limited time
 Narrow scope
• PhD student
 Ample time
 Broader scope
Determine SCOPE

19. SYSTEMATIC LITERATURE REVIEW
HOW TO DO SLR?
1 6
How I did???

20. SYSTEMATIC LITERATURE REVIEW
1 7
RECAP

21. SYSTEMATIC LITERATURE REVIEW
1 9
PRACTICAL CHALLENGES
• It is difficult to draw meaningful conclusions
 Due to ambiguous results, varying outcomes
 Findings are too broad
 Too incomparable
• Hard to screen and filter papers
 Papers not written in same uniform fashion
• Qualitative vs. quantitative evidence
 Comparison in qualitative evidence is difficult
 Comparison in quantitative evidence is easy

22. SYSTEMATIC LITERATURE REVIEW
2 0
PRACTICAL CHALLENGES
• Cannot avoid reporting bias, but can reduce
• Types of bias faced:
 Citation bias (most cited papers given priority)
 Findings bias (negative findings ignored and positive findings taken)
 Publication bias
“Bias is defined as the deviation of results from the truth”
109 biases are introduced in the following study
Choi, B. C., & Pak, A. W. (2005). Bias, overview. Encyclopedia of biostatistics.

23. SYSTEMATIC LITERATURE REVIEW
2 1
SUGGESTIONS
SLR is boring & tiring job
Need to be patient and just keep doing it until published successfully.
Garbage in – garbage out:
Remember that inclusion/exclusion criteria play important role in writing useful review
Inclusion/Exclusion criteria work as cleaning/filtering tool.
Critical analysis achieve citations
Try to find critical points in existing literature and highlight in your SLR
Potential future directions
Try to find what existing researchers say about future. Collect it and report it.

24. SYSTEMATIC LITERATURE REVIEW