2.
Substantive significance
Value, effect, and amount or quantity.
We will discuss: effect size, precision, practical
significance, and clinical significance.
Each is a different part of this concept of importance
of evidence.
3.
A measure of importance that is different than
statistical significance.
Statistical significance is important, but can be
flawed. No matter how low a p-value can be, it does
not tell you how important the finding is.
Trivial changes in data can be statistically, but not
practically significant.
The effect size can tell you the size or magnitude of a
difference or change.
To use effect size, the measures and design of a
study still have to be valid, reliable, and rigorous.
4.
Usually used when means of two or more groups
are compared.
A way to standardized the size of the difference between
group means, translating the difference into standard
deviation units that are interpretable regardless of the
nature of the original scores.
Basically use z-scores and average standard deviations
for the groups being compared.
A rule of thumb: .2=small effect, .5=medium effect, and
.8=large effects.
5.
The r2 is used as a measure of effect size in
correlation research.
Again, we can have a significant correlation (linear
relationship) without an important correlation.
The textbook example is a perfect example, with a
low Pearson r value with a p-value of <.01.
From your reading, you can get a sense that
correlations, even when strong, can leave a lot of the
variance in the data to other potential causes.
6.
An odds ratio compares the odds (probability
measure) of an event or outcome in one group
with the odds of the same event occurring in
another.
So, an odds ratio greater than 1.0 means that an
event/change is more likely to occur in one group
over another. If it is less than 1.0, then it is less likely.
This is a bit more confusing of a measure, but can be
used to look at differences beyond averages and
correlations.
7.
The N-of-1 study is extremely important to all of us as
clinicians, and will be the focus of our final project.
Generally, it is a time series analysis, looking at the change of
an individual over time, compared to a baseline period. Visual,
descriptive analysis.
Guyatt, et al. (2000) called this the highest level of evidence for
an individual client, especially when steps are taken to increase
validity.
Can look at the percentage of change of a behavior between
baseline to treatment (difference between mean of baseline and
mean of treatment).
8.
All measures have error. Thus, the only sure
way to deal with this issue is to measure the
behavior multiple times.
Confidence intervals
Find the standard error measure of the mean (SEM)
and construct an interval around the mean. (so
mean+/-the SEM).
These will be used in Metanalyses.
9.
What is the actual impact of a body of research
on patient care?
Efficacy vs. Effectiveness (outcomes) research.
How important are certain kinds of measures?
With the kinds of research that you are exploring,
there may be very early stage studies that may be
quite helpful toward making treatment decisions,
but must be critical.
10.
Think about practical significance is a reminder
of the need to evaluate the impact of treatment
on the client’s ability to participate fully in
his/her life.
Must look at measures beyond the clinic.
Social validity measures.
Valid and reliable measures of patients’
progress are key.
