Robin Sherman's slides from May 26, 2011 ASBPE webinar, B2B Ethical Struggles and Solutions in a New-Media Era

1. Repor&ng
on
Research
 The
Ethical
Obliga&ons
of
the
Editor
Robin
Sherman,
Editorial
&
Design
Services
RobinSherman@bellsouth.net

2. Agenda
 Ethical
cornerstones
 Goal
of
research
 A
few
sta&s&cal
basics
 Sins
of
unethical
repor&ng
on
research
 Example
 Components
of
a
minimally
good
methodology
explana&on
 What
editors
must
do
 Resources
2
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

3. From
ASBPE’s
Guide
To
Preferred
Editorial
Prac&ces
Sec&on
II.
Standards
for
Editorial
Opera&ons
D.
Research
In
the
case
of
any
research,
rankings,
“best
of”
or
“worst
of”
awards,
buyers’
guides
or
similar
editorial
products,
a
neutral
third
party,
such
as
a
consulIng
firm,
may
be
used
to
help
avoid
conflicts
of
interest.
In
any
editorial
content,
a
clear
and
complete
discussion
of
the
methodology,
including
methodological
and
analy&cal
limita&ons,
should
be
published
to
allow
the
reader
to
make
informed
judgments
about
the
value
of
the
content.
Sec&on
III.
Graphics
and
Photography
B.
Charts
and
Graphics
Informa&on
graphics
should
include
an
explana&on
of
research
methodology
and
give
the
source.
This
applies
as
well
to
graphics
obtained
from
third
parIes.
Why
do
we
need
these
guidelines?
3
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

4. Ethical
cornerstones
 Accuracy
 AMribu&on
 Context
 Proper
emphasis
 Transparency
4
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

5. Research
goal
1)
To
whom
do
you
want
1)
The
theore@cal
popula@on
to
generalize?
2)
What
popula&on
2)
The
study
popula@on
can
you
get
access
to?
3)
How
can
you
get
access
3)
The
sampling
frame
to
them?
4)
Who
is
in
your
study?
4)
The
sample,
sub-­‐sample
5
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

6. A
few
sta&s&cal
basics
 Survey
Responses
needed
for
95%
Confidence
with
±5%
Margin
of
Error
Sample
size
considera&ons
 Popula&on
Size
Responses
Needed
10
10
 Margin
of
error
5%
is
the
commonly
accepted
choice.
The

100
80
margin
of
error
is
the
amount
of
error
that

200
132
you
can
tolerate.

300
169
 Confidence
level

400
197
Typical
choices
are
90%,
95%,
99%.
Measures
the
probability
that
you
can

500
218
project
survey
findings
onto
the
enIre

600
235
universe.
s.

700
249
 Popula&on
size

800
260
How
many
people
are
there
to
choose
your
random
sample
from?
The
sample
size

900
270
doesn't
change
much
for
populaIons
larger

1,000
278
than
20,000.

2,000
323

5,000
357

10,000
370

20,000
377

100,000
383

1,000,000
384
6
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

7. The
Raosoa
Sample
Size
Calculator
hMp://raosoa.com/samplesize.html
7
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

8. A
few
sta&s&cal
basics
 P
(probability)
value
Less
than
.05
usually
considered
sta&s&cally
significant
8
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

9. Sins
of
unethical
repor&ng
on
research*
 1)
Accentua&ng
the
posi&ve
and
ignoring
the
nega&ve
 2)
Generalizing
from
anecdotes
 3)
Not
asking
for
the
evidence
 4)
Wrong
or
insufficient
interpreta&on
of
numbers
 5)
Ignoring
conflicts
of
interest
 6)
Offering
misleading
results
 7)
Ignoring
the
holis&c
picture
and
failing
to
recognize
the
conclusions
and
weaknesses
of
scien&fic
studies
 8)
Ignoring
the
certainty
of
uncertainty
in
science
*Ethics
in
science
journalism,
George
Claassen,
University
of
Stellenbosch
9
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

10. Example
—
Poor
sample
size
10
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

11. Example
—
Exaggerated
claims
11
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

12. Components
of
a
minimally
good
methodology
explana&on
 Invita&ons
were
emailed
to
4,847
people
on
a
list
of
ASBPE
members
and
non-­‐member
business-­‐to-­‐business
(B2B)
editors.
Between
November
2
and
December
10,
2009,
338
people
answered
the
survey
for
a
response
rate
of
7%.
 An
ini&al
screening
ques&on
eliminated
48
people
who
said
they
were
not
staff
editors
or
writers
for
a
business-­‐to-­‐business
print
or
digital
publica&ons.
Among
the
290
who
indicated
they
were
B2B
staff
editors
or
writers,
17
stopped
answering
the
survey
aaer
the
ini&al
screening
ques&on.
The
results
are
based
on
the
273
remaining
B2B
staff
editors
and
writers.
 The
95%
confidence
level
was
used
in
this
study.
Results
labeled
as
sta&s&cally
significant
have
a
5%
or
less
chance
that
they
could
be
aMributed
to
sampling
error
(drawing
a
oddball
sample).
 The
survey
ques&ons
received
between
203
and
269
non-­‐missing
responses
(e.g.
people
who
answered
“don’t
know”
or
lea
a
ques&on
blank
are
missing
responses).
 Percentages
from
these
ques&ons
have
a
margin
of
sampling
error
from
±6.0%
(for
ques&ons
with
269
responses)
to
±6.9%
(for
ques&ons
with
203
responses).
So
using
a
±7%
margin
of
sampling
error
is
safe
for
all
ques&ons.
 This
means
that
sampling
error
should
cause
no
more
than
a
7%
difference
between
the
results
in
our
study
and
the
true
value
in
the
universe
in
95%
of
samples.
 For
example,
our
finding
that
81%
of
B2B
editors
received
a
day
or
less
of
digital
training
in
the
past
year
has
a
95%
chance
of
falling
in
the
interval
between
74%
(81%
–
7%)
and
88%
(81%
+
7%)
in
the
universe
from
which
the
sample
was
drawn.
12
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

13. From
Reynolds
Na&onal
Center
for
Business
Journalism
research
Business
Journalists
study
 The
informaIon
contained
in
this
report
is
based
on
473
in-­‐depth
interviews
with
business
journalists
throughout
the
United
States.
Interviewing
was
conducted
between
April
19
and
May
6,
2010,
by
professional
interviewers
at
BRC’s
state-­‐of-­‐
the-­‐art
Computer-­‐Assisted
Telephone
Interviewing
(CATI)
facility
in
Phoenix.
Interviewing
was
conducted
under
the
direct
supervision
of
BRC
supervisory
personnel,
who
randomly
monitor
interviews
as
they
are
conducted
and
who
validate
completed
interviews.
Prior
to
beginning
the
interviews,
all
interviewers
were
trained
in
all
aspects
of
this
job,
including
protocols
for
open-­‐ended
quesIons
and
sampling
techniques.
 When
reviewing
the
results
from
this
survey,
it
should
be
kept
in
mind
that
all
surveys
are
subject
to
sampling
error.
Sampling
error,
simply
stated,
is
the
difference
between
the
results
obtained
from
a
sample
and
those
that
would
be
obtained
by
surveying
the
enIre
universe
under
consideraIon.
The
overall
sampling
error
for
this
survey
is
approximately
+/-­‐4.6
percent
at
a
95
percent
confidence
interval.
13
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

14. What
editors
must
do
(at
minimum)
 Is
sample
random
and
adequate?
 Is
confidence
level
adequate?
 Is
margin
of
error
is
adequate?
 Is
P
value
is
adequate?
 Ask
a
3rd
party,
independent
sta&s&cian/researcher
to
review
methodology.
 With
your
ar&cle
about
the
research,
include
as
much
info
about
the
methodology
to
give
reader
minimum
way
to
judge
validity.
 AND
explain
the
above
to
your
readers.
14
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

15.  Follow
ASBPE
guidelines.
 Don’t
publish
ar&cles
about
research
with
poor
methodology.
 Don’t
misinform
your
readers.
 Don’t
sin.
15
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

16. Addi&onal
resources
 American
Cancer
Society
hMp://pubs.acs.org/userimages/ContentEditor/1218054468605/ethics.pdf
 Australian
Press
Council
www.rjionline.org/media-­‐accountability/code-­‐of-­‐ethics/
apc-­‐guidelines-­‐for-­‐repor&ng
 American
Educa&onal
Research
Associa&on
www.aera.net/uploadedFiles/Publica&ons/Journals/Educa&onal_Researcher/
3506/12ERv35n6_Standard4Report%20.pdf
 Blackwell
Publishing
hMp://www.blackwellpublishing.com/publica&onethics
 Survey
SoJware
Success,
Jeffrey
Henning,
Founder
&
VP,
Strategy,
Vovici
hMp://www.vovici.com/_assets/pdf/ebook/SurveySoawareSuccess.pdf
 Web
Center
for
Social
Research
Methods
hMp://www.socialresearchmethods.net
16
Covering
Research:
Obliga&on
of
the
Editor
Robin
Sherman
Editorial
&
Design
Services
5/25/11

17. Repor&ng
on
Research
 The
Ethical
Obliga&ons
of
the
Editor
Robin
Sherman,
Editorial
&
Design
Services
RobinSherman@bellsouth.net