Researcher KnowHow session 1 of 3 presented by Ruaraidh Hill PhD MSc FHEA Lecturer in evidence synthesis and Michelle Maden PhD MAFHEA Postdoc research associate in evidence synthesis at the University of Liverpool on 22nd November 2021.
Getting started with a systematic review: developing your review question
1. Getting started with a
systematic review:
developing your
review question
Session 1/3
#systematicreview
@LivUniKnowHow | @LivUniLRiG
2. Ruaraidh Hill PhD MSc FHEA
Lecturer in evidence synthesis
Michelle Maden PhD MA FHEA
Postdoc research associate in evidence synthesis
linktr.ee/RuaraidhHill
3. Your comfort & safety
Slides ✓
Recording ✓
Chat ✓
Practical activity on screen
Evaluation see link in chat & follow-up e-mail
References see final slides
4. This session
Introduction – issues with research evidence
Reviews – overview of systematic reviews | types of
review in the evidence ecosystem
Developing review questions
Reviews – planning next steps
5
6. WeMap
LRiG – Health Data Science, IPH
Liverpool Reviews and Implementation Group (LRiG) – 20+ years
at the core of evidence-based guidance development at NICE.
LRiG delivers a range of high-quality Health Technology
Assessments and capacity development initiatives.
Capacity development – support for individuals and teams to use
evidence to develop practice, services and research.
@LivUniLRiG | liverpool.ac.uk/LRiG
17. Quality – selecting evidence
Instead of just mooching through the
research literature, consciously or
unconsciously picking out papers here and
there that support your pre-existing beliefs,
you take a scientific, systematic approach
to the very process of looking for
scientific evidence, ensuring that your
evidence is as complete and representative
as possible of all the research that has ever
been done
Bad Pharma. Goldacre (2013) Fourth
Estate, London.
19
https://www.youtube.com/
watch?v=RKmxL8VYy0M
24. Review approaches – different questions
• To present a concept
Descriptive
• Explore preferences about what should
happen
Normative
• Investigate a relationship between two or
more variables
Observational/relational
• Investigate the effect of one or more independent
variables on one or more outcome variables
Causal
• Explore factors that cause a condition,
event or process
Theoretical
25. Review approaches – examples (1)
• What is the nature (volume, topic coverage, study
designs) of the literature on online postgraduate training
in English law (2000-to present)?
Descriptive
• What are the views and experiences of people
using a internet banking services?
Normative
• Are adults with disabilities at an increased risk of
violence compared to those without disabilities?
Observational/relational
• What are the effects of fluoridation of drinking water
supplies on the incidence of caries (tooth decay)?
Causal
• What are the links between crime, fear of crime, the
environment and health and wellbeing? How might these
be explained?
Theoretical
28. This session
Introduction – issues with research evidence
Reviews – overview of systematic reviews | types of
review in the evidence ecosystem
Developing review questions
Reviews – planning next steps
33
31. Asking the right question
“To ask the right question is already half the solution of a problem” Carl
Jung, 1959
https://www.azquotes.com/quote/879797
33. Questions about questions – for example:
Who Who is asking the question?
Whom About whom? Who is the question about?
What What is the choice to be made?
Where In what setting is the choice to made?
When
When is the choice to be made? Is the question
about a particular level of service or stage?
Why Why is the question being asked?
38
Deborah
Cohen
bit.ly/2EqULGF
34. Question perspectives – who’s asking?
Policy What should commissioners of services fund?
Practice
What should providers of services do (assess,
measure, offer, follow-up)?
Research
What is the (difference in) level, rate,
preferences, inequalities, effectiveness, cost-
effectiveness (between alternatives)?
Person
What are my options? What are the risks and
benefits? What should I accept?
39
Ulysses
36. Q framework – Who, what, how
Who
Who is the focus?
Specific health condition or age,
sex/gender, occupation socioeconomic,
ethic groups?
What
What is the intervention?
What is idea is being ‘tested’?
An exposure, hypothesis or description of a
phenomenon?
How
How does the ‘what’ affect ‘who’?
Effects, outcomes…
41
37. Q framework – WWH & MindMap
See also Table 2.3/ Doing a Review 2nd Ed
38. Q framework – Who, what, how…
Who Doctors in acute settings
What Face-to-face communication methods/ techniques
How
Quality of doctor – patient communication
(knowledge, attitudes, behaviours, health)
43
40. Question framework – PICO-Sst…
Population -
Intervention -
Comparator -
Outcome -
Study design -
Setting, timing -
45
41. Formulated question – PICO
P
I
C
O
Children and adolescents before
dental treatment…
Pain relief medication
Pain…
satisfaction, cost, completion,
adverse effects
Placebo or no medication
42. PICOSs – example
Review objective
To assess the effects of preoperative analgesics for intraoperative or postoperative pain relief (or both) in children
and adolescents undergoing dental treatment without general anaesthesia or sedation.
Participants/ population (setting)
Children and adolescents up to 17 years of age before dental treatment without GA/sedation (international)
Intervention(s)
Analgesics [pain relief medications] given before dental treatment
Comparator
Placebo or
No analgesic
Outcomes
Postoperative pain
•Intraoperative pain, Preoperative and postoperative anxiety measures, Patient satisfaction, Parental satisfaction,
Cost, Completion of treatment, Adverse events
Study design
Randomised controlled clinical trials http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008392.pub3/full
44. Question framework – PEO
Population
Exposure
The exposure relates to a defined event, activity,
experience or process that groups are exposed
to (and others are not)
Outcome
49
45. Question frameworks – check our learning (1)
For the following systematic review description:
Choose a framework
Identify
Population, intervention , comparator (if applicable), outcome(s)*
Think about
Settings, timing or study design to include
Review approach
46. Question frameworks – check our learning
Effectiveness of public
health measures in
reducing the incidence of
covid-19, SARS-CoV-2
transmission, and covid-19
mortality: systematic
review and meta-analysis
https://doi.org/10.1136/bmj-2021-068302
47. Question frameworks – check our learning
Effectiveness of public health
measures in reducing the incidence of
covid-19, SARS-CoV-2 transmission,
and covid-19 mortality
Population?
Comparator?
Study designs?
https://doi.org/10.1136/bmj-2021-068302
49. Question frameworks – check our learning (2)
For the following systematic review description:
Choose a framework
Identify
Population, intervention or exposure, comparator (if applicable), outcome(s)*
Think about
Settings, timing or study design to include
Review approach
51. Question framework – PICo
Population
What are the characteristics of the person,
patient, group or population?
Interest
The phenomena of interest relates to a defined
event, activity, experience or process
Context
Context is the setting or other distinct
characteristics
60
52. Formulated question - PICo
Co
P
I
Services that test for
Hepatitis C
People who inject drugs
Views, experiences
and attitudes of
hepatitis C testing and
diagnosis
55. Formulated question - SPICE
64
S
P
I
C
E
Services that test for
Hepatitis C
People who inject drugs
None
Hepatitis C testing and
diagnosis
Views, experiences
and attitudes
56. Question frameworks – check our learning (4)
For the following systematic review description:
Choose a framework
Identify
Population; interest, intervention or exposure (as applicable); outcome(s)*
Think about
Settings, timing or study design to include
Review approach
57. Question frameworks – check our learning
Barriers and facilitators to the uptake
of tuberculosis diagnostic and
treatment services by hard-to-reach
populations in countries of low and
medium tuberculosis incidence: a
systematic review of qualitative
literature
58. Question frameworks – check our learning
Barriers and facilitators to the uptake
of tuberculosis diagnostic and
treatment services by hard-to-reach
populations in countries of low and
medium tuberculosis incidence
SPICE
Study designs?
59. Question frameworks – overview
WWH:
Who | What | How
For early thinking
PICO:
Population | Intervention
Comparator | Outcome
For choices between/ comparing
alternatives
(A versus B)
PICo:
Population
Interest | Context
For any question, including
experiences, descriptive reviews
or scoping
Others: SPICE, PROGRESS+
For reviews of qualitative studies
or equalities (e.g.)
69
60. Question frameworks – overview
WWH:
Who | What | How
For early thinking
PICO:
Population | Intervention
Comparator | Outcome
For choices between/ comparing
alternatives
(A versus B)
PICo:
Population
Interest | Context
For any question, including
experiences, descriptive reviews
or scoping
Others: SPICE, PROGRESS+
For reviews of qualitative studies
or equalities (e.g.)
70
61. Question frameworks – overview
71
• Watch from around 40 minutes for
• PICo and SPICE frameworks
• https://stream.liv.ac.uk/w93ffxxu
62. Research questions - key points (1)
• A good research question is:
Clear
Well defined
Appropriate
Manageable
Relevant
63. Research questions - key points (2)
Question frameworks can help develop a good review question
Some notes:
Interventions can be a range of choices
Interest can be explored in a range of ways
Different frameworks can be used, to think about
the same research question
different research questions (or aspects of related Q)
different review approaches
different evidence
Use of frameworks supports transparency and contestability
better quality
64. Evidence – consider…
What evidence informs your discipline or area of practice or
research?
What evidence quality issues are relevant to your discipline or
area of practice or research?
think
66. This session
Introduction – issues with research evidence
Reviews – overview of systematic reviews | types of
review in the evidence ecosystem
Developing review questions
Reviews – planning next steps
77
69. Systematic review – getting started
Plan your ‘10 steps’ – as required
Develop your question(s) and review approach(es)
Scope your questions develop questions further
Check your review meets the needs of your research
question or ‘decision problem’
Refine and record in your review protocol
Consider registering on PROSPERO or other register
72. Systematic review – question frameworks
Cochrane Qualitative and Implementation Methods Group guidance series paper 2: methods
for question formulation, searching, and protocol development for qualitative evidence
synthesis (Harris et al 2017)
https://pubmed.ncbi.nlm.nih.gov/29248725/
Formulating the Evidence Based Practice Question: A Review of the Frameworks (Davies
2011)
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3140151/
Developing NICE guidelines: the manual.(NICE 2014)
https://www.nice.org.uk/process/pmg20/chapter/developing-review-questions-and-planning-
the-evidence-review
Our Researcher Know How series resources/tools
TinyURL.com/KnowHowSRtools
73. Support, next steps
How will you use systematic review methods in your discipline,
area of practice or research?
74. UoL library class no:
R853.S94.B68
tinyURL.com/DoingaReview2
Sample chapters, FAQ, videos available online
86
Watch
Explore
Read
Doing a review?
78. This session
Introduction – issues with research evidence
Reviews – overview of systematic reviews | types of
review in the evidence ecosystem
Developing review questions
Reviews – planning next steps
91
79. Additional links from previous Q&A
Open science/ protocol registration options – please check suitability/
remit of each before choosing to register:
• https://www.crd.york.ac.uk/prospero/
• https://f1000research.com/
• https://osf.io/
Researcher Know How HomePage:
• https://libguides.liverpool.ac.uk/researcherknowhow
80. Getting started with a
systematic review:
developing your
review question
Session 1/3
#systematicreview
@LivUniKnowHow | @LivUniLRiG
www.liverpool.ac.uk/LRiG