1. Systematic Approaches to
Literature Reviewing
Dr. Mark Matthews
Student Learning Development
mark.matthews@tcd.ie
Systematic Approaches to
Literature Reviews
Dr. Mark Matthews
Student Learning Development
http://student-learning.tcd.ie
2. Systematic Approaches to
Literature Reviewing
Dr. Mark Matthews
Student Learning Development
mark.matthews@tcd.ie
Finding your way in the
woods
3. Systematic Approaches to
Literature Reviewing
Dr. Mark Matthews
Student Learning Development
mark.matthews@tcd.ie
Finding your way in the
woods
5. Workshop Overview
explain elements of the systematic review process
explore how these might be used or adapted to
support:
- a thesis literature review;
- approaches to keeping up-to-date with the
literature through a PhD
contextualise this within other approaches to
managing and working with the literature
6. 1 Minute Big Picture Interviewer
Interview partner and take notes
7. The Literature Review ?
“Literature reviews …… introduce a topic, summarise the
main issues and provide some illustrative examples.”
from www.eppi.ioe.ac.uk
Agree? Disagree?
8. If they are to be considered a reliable source of
research evidence they should record how the
primary studies were sought and selected and how
they were analysed to produce their conclusions.
Readers need to be able to judge whether all of the
relevant literature is likely to have been found, and
how the quality of studies was assessed.
1 Agree? 5 Disagree? 10
The Literature Review ?
9. 1. What is a Literature Review?
2. Why do we do Literature Reviews?
3. What are the main challenges?
Discussion
10. Literature Matters
From Holbrook et al (2007)
disciplinary
perspective
connection
to findings
coverage
working
understanding
critical
appraisal
scholarliness
literature use
11. Means you need to be:
critical = evaluate what you read
analyse = extract differing information from what
you read
synthesise = show relationships between
studies/sources; differing definitions, concepts,
theories etc.
evaluate = methodological approaches used
(Hart, 2005)
12. Systematic Reviews
a review of a clearly formulated question that uses
systematic and explicit methods to identify, select and
critically appraise relevant research, and to collect and
analyse data from the studies that are included within
the review. Statistical methods (meta-analysis) may or
may not be used to analyse and summarise the results
of the included studies.
from www.sebc.bangor.ac.uk
13.
14.
15. Student Learning
Development, TCD 15
Systematic v.s. Narrative reviews
• Agreed standards
• High degree of focus
• Minimise bias
• Context and differences
• Bias of empirical studies
Strengths of both approaches
16. “For some review topics, however, the strengths of the
SR may turn into weaknesses…do not allow for
comprehensive coverage” but “narrative reviews do
not reveal how the decisions were made about
relevance of studies”
‘Balancing the strengths of systematic and narrative reviews’ (Collins &
Fauser, 2004) p. 103-104.
17. Key features of the systematic review
process you might want to adopt or
adapt–
1. Explicit and transparent methods
2. a standard set of stages
3. Accountable, replicable and updateable
18. Seven steps (of a Cochrane Review)
1. Formulating a problem
2. Locating and selecting studies
3. Critical appraisal of studies
4. Collecting data
5. Analyzing and presenting results
6. Interpreting results
7. Improving and updating reviews
22. 1. Formulating a Problem
Example Questions?
Standard SR question contains
subject- intervention- outcome- comparator
23. Does drinking coffee raise
people’s blood pressure
(when compared to chocolate)?
Standard SR question contains
subject- intervention- outcome- comparator
25. Where do I start?
Previous Theses
Review Papers
Recent Conference Papers
Subject librarian
Supervisor
26. a broad but defined, systematic sweep
Defined search terms – record recall and precision
Defined search arena - e.g. databases, citation indices,
reference lists from primary and review articles, grey
literature, conference proceedings, research
registers, the internet, individual
researchers/practitioners
Other broad search limits, e.g. language, date,
TIPS!
Document the
search protocol
and record what
research was
found
Systematically
manage the
search output,
e.g. using
endnote
Phase 1- Identify the Research
27. Phase 2- Selection
select from research using criteria related to your research
question
Develop inclusion or exclusion statements, these might
relate to study outcomes, research design, methods
used, population worked with etc.
e.g. studies with a mixed population of men and women
e.g. random control trials only
e.g. maximum exposure time of 10mins
TIPS!
Document the
statements
and
their purpose
(might be
pragmatic or
research related)
28. Date Database Keywords Results
1/11/11
2pm
ACM Digital
Library
Role play, role-
play, role
playing, role-
playing,
bodystorming,
prototyping
10 articles
4/11/11 SpringerLink Role play,
Role-play,
Role-
playing,
Fish bowel
5 articles
1 book
Search Log
29. Running the search
1. How many titles and abstract can you check?
2. How easy will it be to decide to accept or reject a
record?
3. Record the reason for rejection for “Excluded studies”
31. 3. Critical appraisal of studies
“Assessing the quality of methodology is a critical part
of the systematic review process”
No standard approach but there are hierarchies
in fields of study
32. Student Learning
Development, TCD 32
3. Critical appraisal of studies
What would be appropriate
to consider when critically
appraising research in your area?
33. 3. Critical appraisal of studies
disciplinary
perspective
connection
to findings
coverage
working
understanding
critical
appraisal
scholarliness
literature use
disciplinary
perspective
connection
to findings
coverage
working
understanding
critical
appraisal
scholarliness
disciplinary
perspective
connection
to findings
coverage
working
understanding
critical
appraisal
scholarliness
literature use
No standard approach but there
are hierarchies in fields of study
34. Student Learning
Development, TCD 34
4. Collect data & analyse
• Evaluate
• Synthesise results of literature review
– Tables to compare
– Descriptive
– Meta-analysis
39. Writing Style
1. Assess the value of the literature
2. Explain the context research takes place
3. Emphasise limitations of existing
research
4. Tell a story
40. Writing Style
Introduction
What I will show you?
Why?
Body
Why this area?
Don’t leave reader to fill gaps
Conclusion
What we have seen?
How this is relevant to research?
42. Comprehensive Literature Review
Coherent synthesis
of past and present
research in the
domain of study
Source: Dr Hazel Hall, Napier University
What are the main
conclusions on
previous research in
this area?
What are the key
areas of debate in this
area?
Which aspects of this
work are of most
relevance to my
study?
What are the key
concepts in this area?
What have been the
main research
questions?
Where is existing
knowledge “thin”?
How is this topic
approached by
others?
Where are the gaps in
literature?
What are the
main
perspectives
on this topic in
previous
research?
Do parallel
literatures exist
for this topic?
Which
discussions?
Which sub-
themes?
Which writers?
Which work is
subject to
challenge?
Who are
these
“others”?
Which existing
work could be
extended?
In which
subject areas
has the topic
been studied?
44. Other ideas and options
Speed reading
Endnote
Databases in my area
Google Alerts
45. SR websites
Centre for Evidence-based Conservation -http://www.cebc.bangor.ac.uk/
Centre for Reviews and Dissemination (medical) -http://www.york.ac.uk/inst/crd/index.htm
Cochrane Collaboration (international- medical) -http://www.cochrane.org/
EPPI-Centre, Institute of Education - http://eppi.ioe.ac.uk/cms/
Social Policy and Social Care -http://www.york.ac.uk/inst/chp/srspsc/index.htm
If you read one article, an example here of a review of the ways studies in reviews are
appraised - http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=521688
If you prefer power-point, how about this one on mixed method reviews
http://www.ccsr.ac.uk/methods/events/challenges/documents/JamesThomasESRCMeth
odologicalchallenges.ppt
46. Systematic Approaches to
Literature Reviewing
Dr. Mark Matthews
Student Learning Development
mark.matthews@tcd.ie
Finding your way in the
woods
