Meaningfulness vs. S.docx

Meaningfulness vs. Statistical Significance
Program Transcript
MATT JONES: Statistical significance deals with the critical
value of a statistic.
And in a certain philosophy, making a determination of whether
the null
hypothesis is rejected or you fail to reject the null hypothesis.
That is statistical

significance.
Meaningfulness is taking that statistic and determining it's
applicability out in the
real world. So all too often, as researchers, we get caught up in
chasing
statistical significance. And certainly while that's part of
statistics, we also want to
focus on the tie back to the real world.
So while we might find statistical significance in a very large
sample, once we
look at the effect that's present there, or the strength of the
relationship, or the
magnitude of the difference, they all might be extremely small.
So while we might
have quote unquote, "highly statistically significant results,
the effect,
relationship, or differences are rather small and almost
meaningless in the
applied real world."
It's important to understand the difference between statistical
significance and
meaningfulness. Because as a critical consumer of research,
when you're
evaluating claims, you want to match up the actual statistics
with the claims
being made. So if a claim is being made of how large an effect
was or how
important something is, you want to go back to the statistical
analysis and see if
that is indeed the case.
So just because something says there was a statistically
significant result doesn't

necessarily mean you can make claims about how profound an
impact that it
might have out in the real world. Because that might be an
overreach of what the
statistics were actually doing. Statistics will tell you only a
small little piece of the
pie. It's up to us as researchers to put the results into a
specific context and
understand that alignment between the statistics and the results
with a possible
misalignment.
History has given us a lot of examples of where statistics
and statements don't
necessarily match up. It's important to have a critical eye when
it comes to
statistical significance and meaningfulness to evaluate results,
not only of other
researchers, but also your own. And when you're making a
claim about an effect
or a relationship or an intervention, we're able to make that
claim with some
degree of authority. So just because you have statistical
significance, i.e. a P
value below 0.01, that doesn't necessarily mean the magnitude
of the effect was
strong where there's a large difference between two groups
when in fact it might
be very small.
©2016 Laureate Education, Inc. 1

The danger in not understanding the difference between
statistical significance
and meaningfulness is our potential to overreach with results.
So in an era of big
data with extremely large sample sizes, it's very common to
find statistical
significance. But when in fact if we look at the results, the
differences might be
minute, the effect size might be very small, the strength of the
relationship might
be very small. So speaking from a purely statistical
standpoint, there might be an
effect that's present. But if we were to change policy and spend
money based
upon those results, it might not be the most practical or logical
way to proceed.
©2016 Laureate Education, Inc. 2
PRACTICE
29
Working With Survivors of
Sexual Abuse and Trauma:
The Case of Angela

Angela is a 27-year-old, Caucasian female, who first came to
counseling to address her history of sexual abuse. She graduated
from college with a BS in chemistry and has since been
employed
by pharmaceutical companies. After obtaining a new job, she
relo-
cated to an apartment in an East Coast city where she knew no
one. Both of Angela’s parents live on the West Coast, and she
has
one younger brother who also lives in a different state. Angela
has
limited contact with both her mother and brother and does not
have any contact with her father. Angela is obese and disclosed
a history of struggling with her weight and eating issues. She
has
few friends, and those she does have live far away.
Angela has a long history of trauma in her life. She was
sexually
abused between the ages of 9 and 21 by her father, sexually
assaulted
at the age of 14 by a classmate in school, and mugged as a
young
adult. There was domestic violence in the home, also
perpetrated by
her father. Angela’s father is considered an upstanding member
of
the community, and he is well liked and respected by others. No
one
in Angela’s family believes that she was sexually abused, and
her
father joined a “false memory syndrome” group and is
outspoken
about that issue. There has been little discussion in her family
about
what took place in the home while she was growing up.

Angela struggled with daily functioning and exhibited symp-
toms of post-traumatic stress disorder (PTSD). She had a
history
of cutting herself and binge eating and displayed some charac-
teristics of borderline personality disorder. Angela also mildly
dissociated when under duress. Angela suffered from depression
and anxiety and had trouble establishing new relationships, both
socially and at work. Although Angela has a stable job and was
able to complete her work each day, at times she became over-
whelmed by her emotions and retreated to the bathroom where
she cried and sometimes cut herself before returning to her
work-
station. Angela relied on writing, artwork, and her cat for solace
SOCIAL WORK CASE STUDIES: FOUNDATION YEAR
30
and comfort. She was also very active outdoors, often hiking,
biking, and going on camping trips by herself. Her goals in life
were to own her own home, lose weight, enjoy relationships
with
others, and find peace with her traumas.
As a result of the abuse she experienced, it was necessary to
begin treatment focusing heavily on establishing trust and a
rela-
tionship with the client. After 1 year of therapy, deeper process
work was being done around her traumas, and she was able to
open up much more. She disclosed more painful experiences to
the therapist and began expressing her feelings, including
intense
anger at her family members.

Angela also joined a group for survivors of sexual violence in
the same program where she was receiving individual therapy.
She was thus able to meet other survivors and engage them in
relationship building and obtain support. Over time, she lost
100 pounds and made new friends, and her level of functioning
increased dramatically. Six months into the group, however, I
noticed boundary issues between the members of the group and
the group facilitator. After speaking with the group facilitator
about these concerns and others regarding her clinical judgment
and boundary crossing, the decision was made to terminate her.
As a new group facilitator began engaging the group, I noticed
that Angela was not sharing as much in her individual sessions
and, overall, seemed guarded. I tried on numerous occasions to
address the shift, and while Angela acknowledged that trust had
become an issue, she would not directly express her concerns or
feelings. After some discussion, I explained to Angela that
while I
could not discuss the issues concerning the group facilitator,
she
should feel free to talk about her feelings and concerns in
general.
However, it became obvious that trust could not be rebuilt,
partic-
ularly in light of the professional boundary issues with the
group
facilitator. I asked if she wanted to terminate counseling with
me
and find a new therapist, and Angela agreed. I provided Angela
with three referrals so that she could continue her treatment. I
learned that Angela and the former group facilitator had become
friends and remained so after both had left the program in their
respective capacities.

APPENDIX
105
activities as Veronica did. She went to school every day and did
not appear very different from other children in her area. It is
important to note that families in poverty-stricken countries like
Guatemala are deceived by traffickers who offer them money
equivalent to a year’s income in exchange for their children. All
the details of this case are not clear as of yet but it is believed
that
the maternal aunt was working in conjunction with someone
else.
9. Describe any additional personal reflections about this case.
This was a hard case to digest. It is one of those cases that you
end up taking home with you in your heart. This 13-year-old
girl has been through a lifetime of exploitation. For the first few
weeks she would just look at me as if she were looking right
through me. She needed a lot of coaxing to participate.
Although
she still has a great deal of healing ahead of her, Veronica is in
a
much better place and is making every effort to live a normal
life.
Working With Survivors of Sexual Abuse and Trauma:
The Case of Angela
1. What specific intervention strategies (skills, knowledge,
etc.) did you use to address this client situation?
Knowledge of trauma and child sexual abuse was key as was
active listening, validation, boundary setting, and, at times,

confrontation.
2. Which theory or theories did you use to guide your practice?
I applied relational, cognitive behavioral, empowerment, and
strengths-based theories.
3. What were the identified strengths of the client(s)?
Angela’s strengths were her ability to persevere and be resil-
ient, as well as her ability to find time for self-reflection and
self-care. Despite everyone around her telling her otherwise,
she was still able to stand firm in the knowledge that she was
sexually abused and therefore needed to have clear boundaries
with those who did not believe her.
4. What were the identified challenges faced by the client(s)?
Angela’s challenges included an occasional inability to
function
at work, self-harm, and isolation.
jjones2
Typewritten Text
Reflection Questions
jjones2
Typewritten Text
jjones2
Typewritten Text
jjones2
Typewritten Text

SOCIAL WORK CASE STUDIES: FOUNDATION YEAR
106
5. What were the agreed-upon goals to be met to address
the concern?
The goals were to increase functioning, enhance ability to
create and sustain relationships with others, reduce isolation,
address and increase self-esteem, refrain from cutting, and
work through early sexual trauma.
6. What local, state, or federal policies could (or did) affect
this situation?
The statute of limitations in both civil and criminal cases
affected Angela’s case.
7. How would you advocate for social change to positively
affect this case?
I would advocate with legislators in the state to eliminate the
statute of limitations so that survivors of sexual abuse could
prosecute and/or sue their perpetrator when they were ready.
8. Were there any legal or ethical issues present in the case?
If so, what were they and how were they addressed?
There were ethical issues regarding boundaries and dual
relationships. The group facilitator in this case was inappro-
priate with her clients and became personal friends with this
particular client along with the other women in the group. I
addressed this by trying to work with the group facilitator, as
well as by encouraging her to discuss this in her off-site clin-
ical supervision. Because no change was occurring, eventu-
ally the group facilitator was terminated.

9. How can evidence-based practice be integrated into this
situation?
The use of a sequenced, titrated approach using relational
theory to address complex PTSD is incredibly helpful, espe-
cially for those survivors of sexual trauma with multiple
victimizations and difficulty with daily functioning.
10. Describe any additional personal reflections about this
case.
As the individual therapist, this case was heartbreaking for
me. The relationship and trust I had built with this client was
destroyed, and I was placed in a very precarious position. The
client did not want to discuss the changing dynamic and had
jjones2
Typewritten Text
APPENDIX
107
clearly been influenced by the group facilitator, who was
incredibly
friendly and outgoing. There was no other choice but
termination,
and the realization that the damage could not be repaired was
disappointing. However, had I disclosed “my side” of what was
happening, I would have been making the same errors as the
group
facilitator and involving myself in a dysfunctional and
unhealthy

dynamic, including crossing boundaries—exactly what survivors
do not need. There are times when you must “swallow your
pride”
to do what is right and best for the client, especially given the
different variables and considering the ethical issues at play.
Working With Survivors of Sexual Abuse and Trauma:
The Case of Brenna
1. What specific intervention strategies (skills, knowledge,
etc.) did you use to address this client situation?
I used reflective listening and reframing to assist Brenna in
setting goals and determining her unmet needs. I used knowl-
edge of local systems and social service agencies to provide
referrals and to secure needed services.
2. Which theory or theories did you use to guide your practice?
I utilized systems theory.
3. What were the identified strengths of the client(s)?
Brenna’s strengths were her resiliency and self-sufficiency.
Brenna viewed her desire to provide a better future for her
child as a strong motivating factor for changing her life.
4. What were the identified challenges faced by the client(s)?
Brenna lacked a familial support system and network of
friends,
and she was socially isolated. Upon entry to the shelter, she
lacked
medical care, employment, income, and housing. Brenna also
strug-
gled with difficulty reading and writing. Brenna had
experienced
trauma and violence in her past and would be raising her child

alone.
5. What were the agreed-upon goals to be met to address the
concern?
Brenna and I agreed to secure medical care, a housing plan,
and a source of income. Brenna also set goals to improve her
mental health.
jjones2
Typewritten Text
jjones2
Typewritten Text
AMERICAN STATISTICAL ASSOCIATION RELEASES
STATEMENT ON
STATISTICAL SIGNIFICANCE AND P-VALUES
Provides Principles to Improve the Conduct and Interpretation
of Quantitative
Science
March 7, 2016
The American Statistical Association (ASA) has released a
“Statement on Statistical Significance
and P-Values” with six principles underlying the proper use and

interpretation of the p-value
[http://amstat.tandfonline.com/doi/abs/10.1080/00031305.2016.
1154108#.Vt2XIOaE2MN]. The ASA
releases this guidance on p-values to improve the conduct and
interpretation of quantitative
science and inform the growing emphasis on reproducibility of
science research. The statement
also notes that the increased quantification of scientific research
and a proliferation of large,
complex data sets has expanded the scope for statistics and the
importance of appropriately
chosen techniques, properly conducted analyses, and correct
interpretation.
Good statistical practice is an essential component of good
scientific practice, the statement
observes, and such practice “emphasizes principles of good
study design and conduct, a variety
of numerical and graphical summaries of data, understanding of
the phenomenon under study,
interpretation of results in context, complete reporting and
proper logical and quantitative
understanding of what data summaries mean.”
“The p-value was never intended to be a substitute for scientific
reasoning,” said Ron

Wasserstein, the ASA’s executive director. “Well-reasoned
statistical arguments contain much
more than the value of a single number and whether that number
exceeds an arbitrary
threshold. The ASA statement is intended to steer research into
a ‘post p<0.05 era.’”
“Over time it appears the p-value has become a gatekeeper for
whether work is publishable, at
least in some fields,” said Jessica Utts, ASA president. “This
apparent editorial bias leads to the
‘file-drawer effect,’ in which research with statistically
significant outcomes are much more
likely to get published, while other work that might well be just
as important scientifically is
never seen in print. It also leads to practices called by such
names as ‘p-hacking’ and ‘data
dredging’ that emphasize the search for small p-values over
other statistical and scientific
reasoning.”
The statement’s six principles, many of which address
misconceptions and misuse of the p-
value, are the following:

1. P-values can indicate how incompatible the data are with a
specified statistical model.
2. P-values do not measure the probability that the studied
hypothesis is true, or the
probability that the data were produced by random chance
alone.
3. Scientific conclusions and business or policy decisions
should not be based only on
whether a p-value passes a specific threshold.
4. Proper inference requires full reporting and transparency.
5. A p-value, or statistical significance, does not measure the
size of an effect or the
importance of a result.
6. By itself, a p-value does not provide a good measure of
evidence regarding a model or
hypothesis.
The statement has short paragraphs elaborating on each
principle.
In light of misuses of and misconceptions concerning p-values,
the statement notes that
statisticians often supplement or even replace p-values with
other approaches. These include

methods “that emphasize estimation over testing such as
confidence, credibility, or prediction
intervals; Bayesian methods; alternative measures of evidence
such as likelihood ratios or
Bayes factors; and other approaches such as decision-theoretic
modeling and false discovery
rates.”
“The contents of the ASA statement and the reasoning behind it
are not new—statisticians and
other scientists have been writing on the topic for decades,”
Utts said. “But this is the first time
that the community of statisticians, as represented by the ASA
Board of Directors, has issued a
statement to address these issues.”
“The issues involved in statistical inference are difficult
because inference itself is challenging,”
Wasserstein said. He noted that more than a dozen discussion
papers are being published in
the ASA journal The American Statistician with the statement to
provide more perspective on
this broad and complex topic. “What we hope will follow is a
broad discussion across the
scientific community that leads to a more nuanced approach to
interpreting, communicating,

and using the results of statistical methods in research.”
About the American Statistical Association
The ASA is the world’s largest community of statisticians and
the oldest continuously operating
professional science society in the United States. Its members
serve in industry, government
and academia in more than 90 countries, advancing research and
promoting sound statistical
practice to inform public policy and improve human welfare.
For additional information, please
visit the ASA website at www.amstat.org.
For more information:
Ron Wasserstein
(703) 302-1859
[email protected]

Meaningfulness vs. S.docx

Recommended

Recommended

More Related Content

Similar to Meaningfulness vs. S.docx

Similar to Meaningfulness vs. S.docx (9)

More from alfredacavx97

More from alfredacavx97 (20)

Recently uploaded

Recently uploaded (20)

Meaningfulness vs. S.docx