2. Senior clinical research associate, Research Department of
Mental Health
Sciences, University College London, London.
Correspondence to: [email protected]
Keywords
Data interpretation, parametric statistical tests, quantitative
research,
statistics
Review
All articles are subject to external double-blind peer review
and checked
for plagiarism using automated software.
Online
Guidelines on writing for publication are available at
www.nursing-standard.co.uk. For related articles visit the
archive and
search using the keywords above.
4 Page 57
Statistics multiple
choice questionnaire
4 Page 58
Read Sarah Holling’s
practice profile on
head injury
4 Page 59
Guidelines on
how to write a
practice profile
Aims and intended learning outcomes
This article aims to provide a useful
3. introduction to common statistical terms and
the presentation of statistics in research articles.
After reading this article and completing the
time out activities you should be able to:
�4Discuss the importance of assessing the
appropriateness of the statistical tests
performed and accurate interpretation
of findings.
�4Recognise the common statistical tests used
in quantitative research.
�4Identify errors in the reporting of statistical
analysis, such as selective reporting and
overestimating the significance of findings.
�4Understand the importance of statistics
in evidence-based knowledge relevant to
your area.
Introduction
Statistics are the methods and techniques used
to collect, analyse, interpret and present data
(Maltby et al 2007). Nurses routinely use
statistics within their practice, for example
when they give health information to patients
about their diagnosis or prognosis and in
discussing the adverse effects of medication
or treatment. However, many nurses may find
understanding the presentation of statistical
data within a research article challenging.
Fear of statistics is common and is usually
linked with anxiety about understanding and
interpreting statistical data and outcomes
(Williams 2010). In health research, statistics
may be used to determine the prevalence
and incidence of illness or establish if a new
treatment is effective.
5. and standard deviation (measure of variance)
for continuous variables, such as age or years
of education (Table 1). Ranges that show
the lowest and highest measures within that
sample should also be provided; for example,
the range for age will show the youngest and
oldest ages within the sample. Where there
are two sample groups, such as the treatment
and control group, it is important to look for
similarities between the two groups to ensure
they are comparable. If the mean scores and
the range of the measures obtained vary
significantly between the two groups, the
samples may not be considered comparable
and this would introduce bias (prejudice) into
the results of the trial.
It is important to understand the
characteristics of the sample within the
study, as this is the population to which the
results apply and will determine whether they
are generalisable to the wider population for that
patient group. However, caution is needed when
generalising results; for example, the results of
a study undertaken in the United States (US)
cannot be generalised to the UK population.
Although there may be similarities between
the two populations, cultural differences
exist. Therefore, the study would need to be
replicated in the UK to see if similar results are
recorded in this population.
Complete time out activity 1
Presenting data
When examining data presented in tables it is
6. worth considering how the information has
been collected. This is because gender and
age are not subjective measures, but other
recorded outcome measures may be, such
as participants’ mood or behaviour, which
may change over time. For example, the
Beck Depression Inventory (Beck et al 1961)
measures the severity of depression in a person.
The measurement of the tool is subjective,
meaning that scores will vary for an individual
over time depending on how he or she is
feeling. However, if the tool has been validated
correctly then the stability of the measure
should have been assessed through thorough
examination of content, comparisons and
factor analysis. The Beck Depression Inventory
is considered a valid and reliable measure of
depression and is widely used across different
population groups.
Using the Body Mass Index (BMI) (Keys
et al 1972) within a population, one would
expect to find a few very low BMI scores and
a few very high BMI scores, but most would be
centred around the mean score. The mean score
can be affected by the extreme values (outliers)
making it higher or lower than expected. For
example, mean income can be affected by a few
highly paid workers even though the majority
are on a much lower wage. For this reason,
social scientists tend to use the median (middle
value) when describing UK household income.
Understanding the spread of data is
important. Figure 1 shows a standard normal
distribution (spread or shape) in red where
7. the mean score is zero, and the middle of the
distribution and standard deviation is one,
meaning that about 68% of the sample have
a value within ±1 of the mean. The two other
distributions shown are skewed (not equally
distributed around the mean), where the mean
of the distribution is not necessarily the middle
– the means are ±4 in this case. The blue line
is a positive right skew, where more cases are
to the left of the distribution. It is right skewed
as the tail extends out further to the right
than expected. The green line is a negative
left skew with more cases to the right of the
distribution and a longer tail out to the left of
the distribution.
1 Locate a
quantitative research
article with figures and
tables used to represent
data from a randomised
controlled trial. Read
the article and examine
the descriptive data
presented for the
intervention and control
samples. Look for
similarities within the
samples; for example,
are the proportion of
males and females in
the two groups the
same? Are the two
groups of similar age
and range of ages? Do
the two samples have
9. when there is no assumption that data are
normally distributed (Greenhalgh 1997).
Statistical signifi cance is more diffi cult to show
with non-parametric tests (Greenhalgh 1997).
Complete time out activity 2
Hypothesis testing and statistical
signifi cance
A statistical hypothesis is an assumption
about a population parameter (value), which
the study will test. This assumption may or
may not be true. The null hypothesis assumes
that changes to the sample result from chance
and that there is no difference between the
two test scores or there is no difference from
zero. The alternate hypothesis assumes that
changes are infl uenced by some non-random
cause. The alternate hypothesis states there
TABLE 1
Common statistical terms
Statistical term Description
Mean To calculate the mean (average) score, take all the
values, add them up and divide the total by the number of
values. The mean score is often thought of as the middle
of a distribution, however this is only true when the
distribution takes certain shapes, for example the standard
normal distribution curve (Figure 1).
Median This is the middle value of the distribution. To
calculate the median, line all the values up smallest to
largest.
For an odd number of values the median becomes the
middle value. For an even number of items the median
10. becomes the mean of the two central values.
Probability The probability is the number of times an event
occurred divided by the total number of times the event
was
attempted. A probability value will always be between 0
and 1. A value of 0 means that the event never occurs
and 1 means that the event always occurs. This is reported
as the P value.
P value This is a commonly reported statistic resulting from
numerous statistical tests. It can be thought of as the
probability of getting this data or result by chance.
Therefore, the smaller the P value the more likely the
hypothesis being tested is true.
Signifi cance
level
This is the level that the P value is taken to be signifi
cant. This is usually 5% (P=0.05), but other values can
be used. The level of signifi cance is decided before
starting any of the statistical tests.
Standard
deviation
The standard deviation gives an idea of the variability of a
sample. The larger the standard deviation the greater
the indication that the sample is spread out around the
mean.
Odds ratio Odds are the probability of an event occurring
divided by the probability of that event not occurring. An
odds
ratio is the comparison of odds for two binary (two
11. categories) outcomes describing their association.
Correlations The most commonly used correlation is the
Pearsons correlation coeffi cient. This is a calculation of
the linear
association between two variables. It varies from -1 to +1,
where a value of 0 indicates no correlation, a value
of -1 indicates that as one value rises the other decreases
and a value of +1 indicates that as one value increases
the other also increases.
FIGURE 1
Distribution curves
Positive skew (right skew)
Standard normal distribution curve
Negative skew (left skew)
-6 -4 -2 0 2 4 6
0.5
0.4
0.3
0.2
0.1
0.0
p48-55w18.indd 50 28/12/2012 10:31
13. would be rejected and the alternate hypothesis
accepted that the mean birth weight is different
from the hypothesised value of 3.4kg (assuming
the two-sided test had been applied). However,
if the P value had been greater than 0.05 then
the null hypothesis would be accepted that
the mean birth weight of babies born on their
due date is not different from 3.4kg. With
P values very close to 0.05, small changes
to the data may be enough to drive it either
2 Locate a
quantitative research
article that describes
the measures used for
the research study. Does
the article state the
validity and reliability
of the measures and
explain how valid they
are to the study’s
population group? If the
measure was originally
designed in the US,
is there evidence of
further research to
show the validity and
reliability of using the
measure with
a UK population?
TABLE 2
Common parametric statistical tests
Statistical test Purpose of test Example of test use
14. One sample
t-test
Compares the mean scores of a parameter
(value) of the sample to a hypothesised
parameter (estimated value).
To test the hypothesis: is the birth
weight of babies born on their due date
equal to 3.4kg?
Paired t-test Compares two population means and
tests that there is no difference between
the two sets of observations. This can
be done in two ways, either by assessing
the change within an individual or by
matching individuals for comparison.
To compare weight before and after
a diet.
Two sample
t-test
Compares two sample means from the
same population.
To compare pulse rate after two
different forms of exercise.
ANOVA
(analysis of
variance)
Tests whether or not the means of
two or more sample groups are equal.
15. ANOVA is a generalisation of the t-test
to allow comparison where two or more
observations are made.
To compare pulse rate after two or
more different forms of exercise.
ANCOVA
(analysis of
covariance)
As above, but allowing co-variates to be
included in the model.
To compare pulse rate after two or
more different forms of exercise, but
allowing for age or gender.
Correlation
coefficient
Measures the strength of association
between two variables. Pearsons
correlation, Spearman’s rank correlation
and Kendall’s tau correlation are most
commonly used.
To assess the relationship between
quality of life and cognition scores.
Regression A mathematical formula is found to
describe the relationship between two
variables, allowing prediction of one from
the other. Multiple regression allows
inclusion of more than one predictor
and identifies the strongest relationship
17. shows the data depicted on the possible range
of the entire scale. The change looks much
more dramatic on the fi rst scale.
Tables
Data can be presented better in a well laid
out table, as tables can provide condensed
information in a precise manner. There is
nothing wrong with presenting data based on
small sample sizes, provided the reader is made
aware of this. A helpful number to look for in
a table is the sample size, usually presented as
n or N. The number of scores reported may
not be the same as the total number of the
sample, indicating some data are missing. Data
summarised in the main text can be diffi cult to
understand and may need to be read more than
once to comprehend fully what is being said.
Relational descriptions (the association
between pre and post-trial measures) may
result in misunderstandings. If the score on
a scale goes up it can mean that the object of
measurement is improving or deteriorating.
It is therefore important to be familiar with
the scales of measurement being used and
their scoring. For example, high scores on the
Quality of Life in Dementia Scale (Logsdon
et al 1999) indicate better quality of life,
whereas high scores on the Mini-Mental State
Examination (MMSE) (Folstein et al 1975)
indicate poorer cognitive ability.
For reported data, there are several factors
to consider, particularly when the sample size
18. is small. For example:
�4Does the range (dispersal of the highest and
lowest scores) of the sample make sense?
Is it possible and reasonable to see these
scores in the population being studied?
Dementia is a disease associated with old
age and a sample population aged either 60
or 65 and above would usually be expected.
It is not, however, uncommon for people
to develop dementia in their forties or
fi fties. Therefore, in a study of people with
dementia showing a lower end of the age
range of 42 years, consideration should be
given to whether the study has included
one or two participants with early onset
dementia (possibly affecting the mean age)
or whether the study specifi cally sought to
recruit from a sample of people with early
onset dementia.
�4Are the numbers what one would expect
to see for the sample used? For a dementia
study recruiting those with mild to moderate
dementia, one would not expect to see
very low cognition scores indicating severe
dementia. The MMSE has cut-off points of
24-20 for mild dementia, 19-10 for moderate
dementia and 9-0 for severe dementia
(Folstein et al 1975). In a study recruiting
people with mild to moderate dementia,
a score range of 24-10 would therefore
be expected.
�4Are the measures related to one another
in the right way? In a dementia study,
there may be more than one measure
FIGURE 2
21. �4Was any data missing (look at n)? If there is
then it is important to identify what has been
done to handle the missing data
or whether the issue of missing values has
been ignored.
�4Is the mean positioned in the middle of
the range? If not, in which direction might
data be skewed? If data are skewed then
it might not be sensible to use standard
statistical tests.
�4How large is the standard deviation?
The standard deviation shows how much
variation or dispersion there is from the
mean (average) score. A low standard
deviation indicates that the scores are close
to the mean, whereas a high standard
deviation indicates that the scores are spread
out over a larger range of values.
The reporting of randomised controlled
trials is guided by a statement drawn up by
the Consolidated Standards of Reporting
Trials (CONSORT) group (Schulz et al 2010).
The CONSORT statement describes the
principles and recommendations for reporting
trial data and ensuring transparency of the
steps taken to collect and collate data for
the trial. Items of particular note within
the statement are the CONSORT flowchart
(Schulz et al 2010, CONSORT 2012) and the
checklist. The flowchart (Figure 3) gives an
indication of the flow of participants through
the trial, from recruitment to the study to the
final follow-up assessments. The checklist
shows all the items that should be reported
throughout a trial to enable a full appraisal of
22. the quality and robustness of a clinical trial.
Complete time out activity 4
Assessing bias
There are several types of bias that can
have an effect on the outcome of a study
(Table 3). Making sure the study is well
3 Locate a
quantitative research
article with figures
and tables used to
represent data from a
randomised controlled
trial. Within the analysis
section, locate the level
of significance that has
been set. This may be
set at P≤0.05, P≤0.01
or P≤0.001 to show
statistically significant
findings. Does the
analysis use other
statistical tests such as
confidence intervals to
show the significance
of results? Now look
within the tables or text
for the results showing
statistically significant
findings. Are the
conclusions based on
the findings accurate?
4 Locate a
quantitative research
23. article with figures and
tables used to represent
data from a randomised
controlled trial. Read
the article and look at
how the statistics are
reported. How helpful
are the figures and
tables in explaining the
results? Has the data
analysed been presented
according to the
CONSORT flowchart?
Do the reported findings
of the study appear
justified by the results
reported?
(CONSORT 2012)
FIGURE 3
CoNSoRT flowchart
Excluded (n = ...)
�4Not meeting inclusion criteria (n = ...)
�4Declined to participate (n = ...)
�4Other reasons (n = ...)
Allocated to intervention (n = ...)
�4Received allocated intervention (n = ...)
�4Did not receive allocated intervention
(give reasons) (n = ...)
Allocated to intervention (n = ...)
�4Received allocated intervention (n = ...)
�4Did not receive allocated intervention
24. (give reasons) (n = ...)
Lost to follow up (give reasons) (n = ...)
Discontinued intervention
(give reasons) (n = ...)
Analysed (n = ...)
�4Excluded from analysis (give reasons)
(n = ...)
Analysed (n = ...)
�4Excluded from analysis (give reasons)
(n = ...)
Lost to follow up (give reasons) (n = ...)
Discontinued intervention
(give reasons) (n = ...)
Assessed for eligibility (n = ...)
Randomised (n = ...)
A
n
al
ys
is
F
o
llo
w
u
26. Blinding a trial, if done successfully, can
eliminate many types of bias. People who
are involved in a clinical trial include the
participants, clinicians, researchers and
the analyst. It may or may not be possible
to blind the participants and clinicians.
For example, in drug trials, placebo drugs
are often offered so that the clinicians and
participants are unaware of which drug
has been prescribed to which participant.
For psychosocial interventions it is often
impossible to deliver a blinded intervention,
as the clinician and participants are aware
that they are delivering and receiving the
intervention being trialled. Assessment
interviews to collect outcome data may,
however, be collected by blinded researchers.
The clinical trial team often has blinded
and unblinded researchers. Unblinded
researchers are responsible for dealing with
the management of data and are allowed to
know which treatment group a participant
has been allocated to, to arrange attendance at
intervention sessions. The blinded researcher
would be responsible for collecting the research
data and does not have any knowledge of
which group the participant is allocated too,
for example the intervention or the control
group. The blinded researcher would therefore
not be influenced by any preconceived ideas
that he or she has about the intervention.
The analyst responsible for assessing the
results of the data collected should remain
27. blinded for as long as possible. This is usually
until at least the main part of the statistical
TABLE 3
Common sources of bias
Type of bias How bias manifests Explanation of how bias is
introduced
Selection Is the research biased by
selection of the sample to
be used?
Previous knowledge of the likely effects of the
treatment intervention may cause a clinician to select
or avoid the recruitment of particular participants.
In case control studies, this may be a selection of
the particular cases to be included and reasons
for exclusion. For a randomised controlled trial
(RCT), selection of particular participants for a
certain treatment may occur, however with rigorous
allocation concealment (blinding) this selection bias
can be reduced if not eliminated.
Ascertainment Does knowledge of the
group assignment by
the person assessing the
outcomes influence the
assessments?
Recording of relevant measurements should be
completed by someone who is unaware (blinded) of the
treatment being received to reduce this type of bias.
This can be mitigated to a certain extent by including
a variable that indicates the level of perception a
researcher has in relation to the treatment received,
28. for example does the researcher think the participant
was in the treatment or control group?
Performance If the participant knows
what treatment he or
she is receiving, does
this affect his or her
performance on the
subject under research?
This is a complicated question and the perception
of taking part in a trial is enough to improve some
scores. If randomisation happens before the baseline
assessments are completed in a RCT, it is difficult
to factor into any analysis whether the participant’s
knowledge of what treatment he or she was going to
receive affected how the individual responded
to the questionnaires.
Publication Are studies that show
significant results more
likely than those that show
non-significant results to
get published?
Publishing non-significant results can be as important
as publishing significant results. Repeated analyses
on the same data will more than likely find something
significant eventually. Linking the planned study
question with the reported outcomes gives an
indication of the intent, and extent to which that
intent has been satisfied.
p48-55w18.indd 54 28/12/2012 10:31
30. whether all women refused to answer a specific
question, must be identified. If this was the
case, then any analysis that does not take this
into account will be biased; simply adding
gender as a covariate may be enough to ensure
this bias is accounted for.
If imputation (inserting values for missing
data) is used, it is important to ascertain
whether the method of imputation was
appropriate. For example, in dementia studies
the use of the last known observation for
a participant would not be appropriate as
this would make the assumption that the
participant is not experiencing the natural
decline associated with this disease, so the
results could be unduly optimistic.
Complete time out activity 5
Conclusion
An understanding of basic statistics will
assist nurses to interpret the findings within a
research article more accurately. It is essential
that nurses have a good understanding of the
significance of findings, which allows them to
assess the validity and reliability of the research
evidence and whether its implementation in
practice is appropriate.
Statistics is a complex subject, but
understanding should improve with familiarity.
The article has provided an overview of
common statistical terms relevant to presenting
statistical data within quantitative research.
Nurses are encouraged to develop their
31. knowledge of statistics and take advantage
of opportunities to use this when appraising
research articles NS
Complete time out activity 6
5 Locate a
quantitative research
article with a detailed
description of the
methods and analysis
used in the research.
Read the research
article and identify what
possible bias or errors
there may be in the
study.
6 Now that you have
completed the article,
you might like to write
a practice profile.
Guidelines to help you
are on page 60.
References
Altman DG, Bland MJ (2007)
Statistics notes: missing data.
British Medical Journal. 334,
7590, 424.
Beck AT, Ward CH, Mendelson M,
Mock J, Erbaugh J (1961) An
inventory for measuring depression.
Archives of General Psychiatry.
4, 561-571.
32. Blessed G, Tomlinson BE, Roth M
(1968) The association between
quantitative measures of dementia
and of senile change in the cerebral
grey matter of elderly subjects.
British Journal of Psychiatry.
114, 512, 797-811.
CONSORT (2012) Consort:
transparent reporting of trials.
www.consort-statement.org (Last
accessed: November 27 2012.)
Folstein MF, Folstein SE,
McHugh PR (1975) ‘Mini-mental
state’. A practical method for
grading the cognitive state of
patients for the clinician.
Journal of Psychiatric research.
12, 3, 189-198.
Greenhalgh T (1997) How to read
a paper. Statistics for the
non-statistician. I: different types of
data need different statistical tests.
British Medical Journal. 315, 7104,
364-366.
Keys A, Fidanza F, Karvonen MJ,
Kimura N, Taylor HL (1972) Indices
of relative weight and obesity.
Journal of Chronic Diseases. 25, 6,
329-343.
33. Le Fanu J (2002) The case of the
missing data. British Medical
Journal. 325, 7378, 1490-1493.
Logsdon RG, Gibbons LE,
McCurry SM, Teri L (1999) Quality
of life in Alzheimer’s disease: patient
and caregiver reports. Journal of
Mental Health and Aging. 5, 1, 21-32.
Maltby J, Day L, Williams G (2007)
Introduction to statistics for nurses.
Pearson Education, Harlow.
McCluskey A, Lalkhen AG (2007)
statistics IV: Interpreting the
results of statistical tests. ceaccp.
oxfordjournals.org/content/7/6/
208.full.pdf+html (Last accessed:
November 27 2012.)
Schulz KF, Altman DG, Moher D;
CONSORT Group (2010) CONSORT
2010 Statement: updated guidelines
for reporting parallel group
randomised trials. British Medical
Journal. 340, c332.
Williams AS (2010) Statistics
anxiety and instructor immediacy.
Journal of statistics Education.
18, 2, 1-18.
Wood AM, White IR, Thompson SG
34. (2004) Are missing outcome data
adequately handled? A review of
published randomized controlled
trials in major medical journals.
Clinical trials. 1, 4, 368-376.
p48-55w18.indd 55 28/12/2012 10:31
NURSING STANDARD january2::vol27no18::2013 57
Learning zone assessment
Statistics: part 2
TEST YOUR KNOWLEDGE AND
WIN A £50 BOOK TOKEN
Thisself-assessmentquestionnaire(SAQ)
willhelpyoutotestyourknowledge.Each
weekyouwillfindtenmultiple-choice
questionsthatarebroadlylinkedtothe
learningzonearticle.Note:Thereisonly
onecorrectanswerforeachquestion.
Ways to use this assessment
��Youcouldtestyoursubjectknowledge
byattemptingthequestionsbefore
readingthearticle,andthengoback
overthemtoseeifyouwouldanswer
anydifferently.
��Youmightliketoreadthearticletoupdate
yourselfbeforeattemptingthequestions.
Prize draw
36. b) Do not always indicate
clinically significant findings o
c) Do not underpin
evidence-based practice o
d) Are always misleading o
3. Descriptive data are also
known as:
a) Summary statistics o
b) Sample size o
c) Sample population o
d) Demographic variables o
4. Which of these equals the mean?
a) Measure of variance o
b) Average o
c) Middle value o
d) Significance o
5. The odds ratio is:
a) Total of all values o
b) Number of times an event
occurred divided by the number of
times the event was attempted o
c) Addition of all values divided by
the number of values o
d) Probability of an event occurring
divided by the probability of that
event not occurring o
6. Which of the following is not a
37. subjective measure?
a) Mood o
b) Behaviour o
c) Gender o
d) Severity of depression o
7. Which test compares two sample
means from the same population?
a) Paired t-test o
b) Two sample t-test o
c) One sample t-test o
d) Correlation coefficient o
8. Common sources of bias in
research are:
a) Selection bias o
b) Ascertainment bias o
c) Performance bias o
d) All of the above o
9. Which letter denotes the size of
the sample?
a) n o
b) p o
c) s o
d) r o
10. Reporting of randomised trials is
guided by:
a) CONSORT o
b) MMSE o
c) NICE o
d) SIGN o
This self-assessment questionnaire
was compiled by Tanya Fernandes
39. STANDARD / RCN PUBLISHING
Learning zone
C O N T I N U I N G P R O F E S S I O N A L D E V E L O P
M E N T
Understanding quantitative
research: part 1
NS673 Hoe J, Hoare Z (2012) Understanding quantitative
research: part 1.
Nursing Standard. 27, 15-17, 52-57. Date of acceptance: March
2 2012.
Abstract
This article, which is the first in a two-part series,
provides an introduction
to understanding quantitative research, basic statistics and
terminology
used in research articles. Critical appraisal of research
articles is essential
to ensure that nurses remain up to date with evidence-based
practice
to provide consistent and high-quality nursing care. This
article focuses
on developing critical appraisal skills and understanding the
use and
implications of different quantitative approaches to research.
Part two
of this article will focus on explaining common statistical
terms and the
presentation of statistical data in quantitative research.
Authors
Juanita Hoe
Senior clinical research associate, Research Department of
Mental Health
40. Sciences, University College London, London.
Zoë Hoare
Clinical trials statistician, Bangor University, Bangor.
Correspondence to: [email protected]
Keywords
Evidence-based practice, quantitative research, statistics, study
design
Review
All articles are subject to external double-blind peer review
and checked
for plagiarism using automated software.
Online
Guidelines on writing for publication are available at
www.nursing-standard.co.uk. For related articles visit the
archive and
search using the keywords above.
4 Page 58
Statistics multiple
choice questionnaire
4 Page 59
Read Joanne Hardy’s
practice profile on
pre-operative assessment
4 Page 60
Guidelines on
how to write a
practice profile
Aims and intended learning outcomes
This article aims to provide information to nurses
who are attempting to appraise and review
41. quantitative research articles critically. The
broad nature of research means it is not possible
to cover all aspects of research methodology in
detail, however the article can help nurses gain a
better understanding of quantitative research and
the principles that underpin it. After reading this
article and completing the time out activities you
should be able to:
�4Acknowledge the importance of identifying,
appraising and understanding quantitative
research evidence.
�4Identify key questions for appraising research
evidence critically.
�4Recognise and identify the common
quantitative research methods used within
different studies.
�4Appraise and evaluate the limitations of
quantitative research evidence from a range
of sources.
�4Develop evidence-based knowledge relevant
to your area of practice.
Introduction
At a time of considerable advances in medical
and nursing practice, it is important that
healthcare professionals stay abreast of the
changes (Department of Health (DH) 1997,
2008). The Code (Nursing and Midwifery
Council (NMC) 2008) states that nurses need
to provide a high standard of care at all times.
Nurses also need to ensure that their skills
and knowledge reflect evidence-based practice
(NMC 2008). The ability of nurses to use or
undertake research is therefore essential, not
p52-57w15-17.indd 52 10/12/2012 10:59
43. whether it is valid and relevant, and how it can
be integrated into care. Evidence-based practice
is viewed as the ‘integration of best research
evidence with clinical expertise and patient
values’ (Sackett et al 2000).
Improvements in clinical judgement can
only be achieved by developing abilities in
deliberate reasoning (logical and reflective
thinking) and analytical skill (Paniagua 1995,
Benner et al 2009). Advances in medical and
nursing research and technology, as well as the
need to stay abreast of current evidence and
best practice, may be daunting for some, but
are vital to ensure best practice. It is important
that nurses do not feel discouraged from
developing and using critical appraisal skills.
Only by using and practising such skills will
nurses become more familiar with quantitative
research methodology and outcomes, and
develop the skills necessary to assess the
quality of published evidence and its relevance
to nursing practice.
Critical appraisal
Nurses may not be in a position to be
involved in research studies or undertake
research themselves. They do, however,
have the capacity to make sense of published
evidence through appraisal of the literature.
Critical appraisal is the process of examining
systematically a research article to assess its
validity – whether the study measures what is
says it measures – results and relevance before
using the evidence to inform decision making
44. (Burls 2009). It is the process of assessing the
facts presented and the quality of the study to
determine best evidence (Fowkes and Fulton
1991, Burls 2009). Nurses should, however,
be aware that not all published studies are of
a good quality and that research may be poorly
reported, weak in design or flawed (Greenhalgh
1997, Churchill 1998, Godshall 2009).
Critical appraisal is therefore a fundamental
component of establishing evidence-based
practice. Sackett et al (1996) suggested that
critical appraisal of the best available and
clinically relevant information is essential
for designing and developing new research,
and integrating it with clinical expertise to
implement evidence-based practice. Nurses
should be able to use their critical appraisal
skills to decide whether the quality of the
research evidence is sufficient to underpin their
practice or whether the evidence is flawed.
Most published research articles go through
a peer-review process, where the credibility of
the article has been assessed by experts from
within the area in which the research was
carried out. The purpose of critiquing is to
analyse a research article, identifying flaws,
evidence of bias or other factors that might have
affected the results and how this, in turn, might
affect the findings or outcome (Godshall 2009).
Complete time out activity 1
Assessing quality
Greenhalgh (1997) identified three key
questions for assessing the methodological
46. STANDARD / RCN PUBLISHING
Learning zone statistics
�4Is the study of interest?
�4Who are the subjects and how were they
recruited?
�4How accurate is the data collected?
�4Are the measures used valid and reliable?
�4Are the statistical methods used appropriate
and performed properly?
�4What did the study find?
�4What are the implications of the study?
There are several online resources available
to help develop skills in critical appraisal and
literature searching, and several journal articles
offer frameworks of questions for assessing
the quality of research studies (Jaeschke et al
1994, Greenhalgh 1997, Greenhalgh and
Taylor 1997, Churchill 1998, Morton and
Morton 2003,
Solution
s for Public Health
2012). Nurses are also encouraged to attend
workshops on how to conduct database
searches to ensure access to the most recently
published literature.
Complete time out activity 2
47. Structure of a research article
Published research articles generally consist
of a standardised layout. This includes
(Greenhalgh 1997):
�4Abstract – the abstract summarises the main
points of the study design, its aim, how the
research was undertaken and key findings
from the research.
�4Introduction – the introduction provides
a comprehensive overview of research
previously undertaken in the area of interest
and specifies why this particular piece of
research is needed. It should be noted that
the background information may be brief at
times, particularly where authors are limited
by word count.
�4Method – the method section should begin
with a description of the aims and objectives
of the study and the hypothesis (research
question) that the study intends to answer.
This is followed by a description of the study
design and the sample population, such as
how many participants were needed and
how they were recruited to the study, and
48. if applicable the randomisation procedures
followed. Where relevant, a description of
the planned intervention should be given,
and any procedures that were followed for
the recruitment of participants, applying
measures, providing interventions and
collecting data should be outlined. This
is necessary to ensure that the study can
be replicated elsewhere. The primary
and secondary outcomes that are being
measured should be identified as well as
the tools (assessments or scales) used to
determine this, with brief reference to their
acceptability, validity and reliability. Scales
are known to measure what they should
measure and this can be done consistently
and without any adverse effects, such as
causing increased burden or distress to
participants. Any ethical issues and details of
who gave permission for the research to be
undertaken should be detailed.
�4Analysis – the analysis should describe the
steps taken to analyse the data collected and
justification for the statistical tests applied.
49. �4Results – the results should include details
of the number of people completing the
study, with an explanation for those that
did not complete the study. Demographic
information, such as age, gender, ethnicity,
living area, education and income should
be provided for the sample population. This
helps to determine whether the results are
generalisable to the wider population or
limited in their application. Descriptive and
comparative data may be presented alongside
figures or tables that help to illustrate any
significant results.
�4Discussion – the discussion covers key
findings of the research, interpreting
their usefulness for clinical practice and
implications for future research. There should
also be a comparison of the findings to other
similar research studies, as this demonstrates
how the findings fit with existing research and
builds on the evidence. The authors should
also reflect critically on the limitations of the
study and any conclusions drawn should be
justifiable and relevant to the results given.
�4Conclusion – the conclusion restates the
51. such research include focus groups, in-depth
interviews and narrative inquiry (Polit and
Hungler 1995, Greenhalgh and Taylor 1997,
Bowling 2002). Qualitative data can be used
to generate ideas (theory) or hypotheses, which
may then be addressed using quantitative
methods (Polit and Hungler 1995).
Quantitative techniques are used to test
hypotheses, determine causation (relationships)
between variables (characteristics or values that
can be changed) and measure the frequency
(number) of observations (Fowkes and Fulton
1991, Greenhalgh 1997, Bowling 2002).
Quantitative data can be counted or measured
and examples of such methods include clinical
trials, surveys and cohort studies.
Quantitative methods have traditionally
been considered more rigorous than qualitative
methods, with randomised controlled trials
(RCTs) and systematic reviews being the ‘gold
standard’ for determining evidence (Sackett
et al 2000). There is an established ranking or
hierarchy of evidence for assessing the quality
52. and robustness of methodological approaches
(Evans 2003), but quantitative and qualitative
methods are both considered valid and
complementary when applied correctly, and
may also be integrated (Bowling 2002).
Research in nursing has focused largely
on qualitative approaches, although there is
now a move towards using mixed methods,
which combine qualitative and quantitative
approaches. As it is beyond the scope of this
article to describe the range of qualitative and
quantitative research methods, the article
focuses on common quantitative research
methods. Qualitative approaches to research
are dealt with elsewhere within the nursing
literature (Ploeg 1999, Holloway and Wheeler
2010, Streubert and Carpenter 2010).
Quantitative approaches
The hierarchy of evidence (weighting of the
strength of the evidence) for quantitative
approaches is generally recognised as set out
below. As previously indicated, systematic
reviews and RCTs are considered the gold
53. standard for determining evidence.
Systematic reviews
Systematic reviews provide an overview of the
evidence relating to a specific research question
by combining data from existing or primary
research, usually published RCTs. Where RCTs
are unavailable, observational studies such as
controlled before and after or interrupted time
series (where outcomes are measured at specific
points in the study) may be included, and the
evidence may also be informed by the findings
of qualitative studies.
The quality of the studies included in the
review is assessed systematically. Where
possible a meta-analysis is carried out in
which numerical data from two or more
clinical studies is pooled and a weighted
average calculated. Systematic reviews and
meta-analysis are used to determine the
effectiveness of healthcare interventions. An
example of a systematic review is Moore and
Cowman’s (2008) review of risk assessment
54. tools for the prevention of pressure ulcers.
Randomised controlled trials
RCTs are experimental studies that are used to
test the effectiveness of interventions between
two or more groups, usually an intervention
and a control (non-intervention or placebo)
group. Participants are randomly allocated
to a group and the intervention is delivered
under tightly controlled conditions to avoid
systematic errors (bias) and random errors
(chance). The study participants and/or
those undertaking the research assessments
may be blinded (single-blind where either
the participant or the researcher is unaware
to which group the participant has been
randomly allocated, or double-blind where
the participant and researcher are unaware of
the allocation), where information about who
is and who is not receiving the intervention
is concealed until the trial is complete. Data
are usually collected before and after the
intervention and differences in outcome
examined between the groups. An example
of an RCT is Kataoka et al’s (2010) study
55. comparing the use of self-administered
questionnaires versus interview as a screening
method for intimate partner violence in the
prenatal setting in Japan.
Cohort studies
Longitudinal or cohort studies are
observational studies of people with common
characteristics or experiences and are used
to determine the prognosis and progress
of disease over time, from its early to late
stages. Data are collected at two or more
points over a particular period, usually of
several years duration. An example of a
cohort study is Mueller et al’s (2010) study of
patient functioning as a predictor of nursing
workload in acute hospital units providing
rehabilitation care.
2 Make a list of
the main differences
between quantitative
and qualitative research.
3 Read the article
57. study of mental health first aid training for
the Chinese community in Australia, which
examined the effects of such training on
knowledge about and attitudes towards people
with mental illness
Case-control studies
Case-control studies focus on people with a
specific diagnosis or disease, who are matched
with people who do not have the disease (the
controls). Data are collected on the two groups
and compared to explore what differences
exist between the groups and identify any
characteristics that may be contributing to the
disease. An example of a case-control study is
Lazovich et al’s (2010) study of indoor tanning
and risk of melanoma.
Cross-sectional surveys
Cross-sectional surveys are used to determine
the frequency of disease or diagnosis (screening),
risk factors or other phenomenon, such as
events, behaviour and attitudes, at one point
in time. Although methodologically weak,
cross-sectional surveys can be used to explore
58. causal relationships (cause and effect) between
variables. An example of a cross-sectional
survey is Aung et al’s (2010) study of access to
and use of GP services among Burmese migrants
in London.
Case studies
Case series and case studies are descriptive
in nature and are used to determine factors
contributing to the development of an illness.
Case studies are considered the weakest level
of evidence, but are useful in the early stages
of research about a particular disease. An
example of a case study is Chaboyer et al’s
(2010) study of bedside nursing handover.
Complete time out activity 4
Ethics
Consideration must be given to the ethical
implications of the research being undertaken.
Any health-related research project involving
humans, their tissue and/or data must be
reviewed and approved by a research ethics
committee before it can start. The
59. research ethics committee will review
the research protocol and other project
documents to ensure that the dignity,
rights, safety and wellbeing of research
participants is protected. Of particular
concern are issues related to participants’
capacity to consent to take part in the
research study, maintaining confidentiality,
and data management and storage.
Stringent guidance is provided by the
Research Governance Framework for
Health and Social Care (DH 2005), which
supports undertaking good quality studies
and promotes good clinical practice within
research. The Mental Capacity Act 2007
also provides guidance on the inclusion of
vulnerable adults in research, such as people
with a learning disability or patients with
dementia, who may not have the capacity
to consent to their participation.
Results
The results are an important part of
a quantitative study as they provide
60. information on the amount of data collected
and the outcome of the statistical analyses
performed. The results reported can be
particularly difficult to follow and may
be open to misinterpretation if the data
are not reported accurately and clearly.
The credibility of the research undertaken
is based on the thoroughness of the data
reported, appropriateness of the statistical
tests performed and accurate interpretation
of the findings. Researchers conducting good
quality clinical studies will have sought advice
and assistance from a qualified statistician
in planning and undertaking the analysis,
and there should be justification given for the
analysis used. The statistical analysis used
should be relevant to the design of the research
proposed and the research question stated.
Conclusion
Staying abreast of developments in health
care, and using this knowledge to inform and
improve patient care, can be both challenging
and rewarding for nursing staff. Key to
this is having good critical appraisal skills
61. and an understanding of quantitative and
qualitative research design and methodological
approaches. While research terminology is not
always easy to comprehend, it does become
easier to understand with familiarity. The
4 Identify a relevant
area in your clinical
practice that would be
of interest to research.
What would be your
research question?
What quantitative
research methods do
you think you could
use to investigate this
area of practice? What
outcomes would you
measure? What support
is available in your
clinical area to help you
undertake a research
study?
p52-57w15-17.indd 56 10/12/2012 10:59
63. quantitative research NS
Complete time out activity 5
References
Aung NC, Rechel B, Odermatt P
(2010) Access to and utilisation
of GP services among Burmese
migrants in London: a cross-sectional
descriptive study. BMC Health
Services Research. 10, 285.
Benner P, Tanner C, Chesla C (2009)
Expertise in Nursing Practice:
Caring, Clinical Judgment, and
Ethics. Second edition. Springer,
New York NY.
Bowling A (2002) Research
Methods in Health: Investigating
Health and Health Services. Second
edition. Open University Press,
Maidenhead.
Burls A (2009) What is Critical
Appraisal? www.whatisseries.co.uk/
64. whatis/pdfs/What_is_crit_appr.pdf
(Last accessed: November 21 2012.)
Chaboyer W, McMurray A, Wallis M
(2010) Bedside nursing handover:
a case study. International Journal
of Nursing Practice. 16, 1, 27-34.
Churchill R (1998) Critical appraisal
and evidence-based psychiatry.
International Review of Psychiatry.
10, 4, 344-352.
Department of Health (1997) The
New NHS: Modern, Dependable.
The Stationery Office, London.
Department of Health (2005)
Research Governance Framework
for Health and Social Care. Second
edition. The Stationery Office,
London.
Department of Health (2008) High
Quality Care for All: NHS Next Stage
65. Review Final Report. The Stationery
Office, London.
DiCenso A, Callum N, Ciliska D
(1998) Implementing
evidence-based nursing: some
misconceptions. Evidence-Based
Nursing. 1, 2, 38-39.
Elliot PA (1995) The development of
advanced nursing practice: 1. British
Journal of Nursing. 4, 11, 633-636.
Evans D (2003) Hierarchy of
evidence: a framework for ranking
evidence evaluating healthcare
interventions. Journal of Clinical
Nursing. 12, 1, 77-84.
Fowkes FG, Fulton PM (1991)
Critical appraisal of published
research: introductory guidelines.
British Medical Journal. 302, 6785,
1136-1140.
66. Godshall M (2009) Fast Facts
for Evidence-Based Practice:
Implementing EBP in A Nutshell.
Springer Publishing Company, New
York NY.
Greenhalgh T (1997) How to read
a paper. Getting your bearings
(deciding what the paper is about).
British Medical Journal. 315, 7102,
243-246.
Greenhalgh T, Taylor R (1997)
How to read a paper. Papers that
go beyond numbers (qualitative
research). British Medical Journal.
315, 7110, 740-743.
Holloway I, Wheeler S (2010)
Qualitative Research in Nursing
and Healthcare. Third edition.
John Wiley & Sons, Chichester.
Jaeschke R, Guyatt G, Sackett DL
67. (1994) Users’ guides to the medical
literature: III. How to use an article
about a diagnostic test: A. Are the
results of the study valid? Journal of
the American Medical Association.
271, 5, 389-391.
Kataoka Y, Yaju Y, Eto H,
Horiuchi S (2010) Self-administered
questionnaire versus interview
as a screening method for
intimate partner violence in
the prenatal setting in Japan:
a randomised controlled trial.
BMC Pregnancy and Childbirth.
10, 84.
Lam AY, Jorm AF, Wong DF (2010)
Mental health first aid training
for the Chinese community in
Melbourne, Australia: effects on
knowledge about and attitudes
toward people with mental illness.
International Journal of Mental
68. Health Systems. 4, 18.
Lazovich D, Isaksson Vogel R,
Berwick M, Weinstock MA,
Anderson KE, Warshaw EM
(2010) Indoor tanning and risk of
melanoma: a case-control study in
a highly exposed population. Cancer,
Epidemiology, Biomarkers and
Prevention. 19, 6, 1557-1568.
Moore ZE, Cowman S (2008) Risk
assessment tools for the prevention
of pressure ulcers. Cochrane
Database of Systematic Reviews.
Issue 3, CD006471.
Morton and Morton (2003)
Evidence Based Nursing Practice.
www.ebnp.co.uk (Last accessed:
November 21 2012.)
Mueller M, Lohmann S, Strobl R,
Boldt C, Grill E (2010) Patients’
functioning as predictor of nursing
69. workload in acute hospital units
providing rehabilitation care: a
multi-centre cohort study. BMC
Health Service Research. 10, 295.
Nursing and Midwifery Council
(2008) The Code: Standards of
Conduct, Performance and Ethics for
Nurses and Midwives. NMC, London.
Paniagua H (1995) The scope
of advanced practice: action
potential for practice nurses.
British Journal of Nursing. 4, 5,
270-274.
Ploeg J (1999) Identifying the best
research design to fit the question.
Part 2: qualitative designs. Evidence
Based Nursing. 2, 36-37.
Polit DF, Hungler BP (1995)
Nursing Research: Principles
and Methods. Fifth edition.
70. Lippincott Williams and Wilkins,
Philadelphia PA.
Rycroft-Malone J, Seers K,
Titchen A, Harvey G, Kitson A,
McCormack B (2004) What counts
as evidence in evidence-based
practice? Journal of Advanced
Nursing. 47, 1, 81-90.
Sackett DL, Rosenberg WM,
Gray JA, Haynes RB, Richardson WS
(1996) Editorial. Evidence based
medicine: what it is and what it isn’t.
British Medical Journal. 312, 7023,
71-72.
Sackett DL, Straus SE,
Richardson WS, Rosenberg W,
Haynes RB (2000) Evidence-Based
Medicine: How to Practice and
Teach EBM. Second edition.
Churchill Livingstone, Edinburgh.
Salvage J (1998) Evidence-based
71. practice: a mixture of motives?
Journal of Research in Nursing. 3, 6,
406-418.