Reporting on Research: The Ethical Obligations of the Editor

2,202 views

Published on

Robin Sherman's slides from May 26, 2011 ASBPE webinar, B2B Ethical Struggles and Solutions in a New-Media Era

Published in: News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,202
On SlideShare
0
From Embeds
0
Number of Embeds
678
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Reporting on Research: The Ethical Obligations of the Editor

  1. 1. Repor&ng  on  Research    The  Ethical  Obliga&ons  of  the  Editor   Robin  Sherman,  Editorial  &  Design  Services   RobinSherman@bellsouth.net  
  2. 2. Agenda    Ethical  cornerstones    Goal  of  research    A  few  sta&s&cal  basics    Sins  of  unethical  repor&ng  on  research    Example    Components  of  a  minimally  good  methodology  explana&on    What  editors  must  do    Resources   2   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  3. 3. From  ASBPE’s  Guide  To  Preferred  Editorial  Prac&ces  Sec&on  II.  Standards  for  Editorial  Opera&ons   D.  Research  In  the  case  of  any  research,  rankings,  “best  of”  or  “worst  of”  awards,  buyers’  guides  or  similar   editorial  products,  a  neutral  third  party,  such  as  a  consulIng  firm,  may  be  used  to  help  avoid   conflicts  of  interest.  In  any  editorial  content,  a  clear  and  complete  discussion  of  the   methodology,  including  methodological  and  analy&cal  limita&ons,  should  be  published  to   allow  the  reader  to  make  informed  judgments  about  the  value  of  the  content.  Sec&on  III.  Graphics  and  Photography   B.  Charts  and  Graphics    Informa&on  graphics  should  include  an  explana&on  of  research  methodology  and  give  the   source.  This  applies  as  well  to  graphics  obtained  from  third  parIes.  Why  do  we  need  these  guidelines?   3   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  4. 4. Ethical  cornerstones    Accuracy    AMribu&on    Context    Proper  emphasis    Transparency   4   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  5. 5. Research  goal  1)  To  whom  do  you  want            1)  The  theore@cal  popula@on   to  generalize?  2)  What  popula&on            2)  The  study  popula@on   can  you  get  access  to?  3)  How  can  you  get  access            3)  The  sampling  frame   to  them?  4)  Who  is  in  your  study?          4)  The  sample,  sub-­‐sample   5   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  6. 6. A  few  sta&s&cal  basics     Survey  Responses  needed  for     95%  Confidence  with  ±5%  Margin  of  Error  Sample  size  considera&ons     Popula&on  Size            Responses  Needed                                10  10    Margin  of  error   5%  is  the  commonly  accepted  choice.  The                            100    80   margin  of  error  is  the  amount  of  error  that                            200                    132   you  can  tolerate.                              300                                169    Confidence  level                            400                              197   Typical  choices  are  90%,  95%,  99%.   Measures  the  probability  that  you  can                            500                              218   project  survey  findings  onto  the  enIre                            600                              235   universe.   s.                                700                              249    Popula&on  size                            800                                    260   How  many  people  are  there  to  choose  your   random  sample  from?  The  sample  size                            900                              270   doesnt  change  much  for  populaIons  larger                      1,000                              278   than  20,000.                      2,000                            323                      5,000                              357                  10,000                              370                  20,000                              377              100,000                            383        1,000,000                              384   6   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  7. 7. The  Raosoa  Sample  Size  Calculator    hMp://raosoa.com/samplesize.html     7   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  8. 8. A  few  sta&s&cal  basics    P  (probability)  value   Less  than  .05  usually  considered  sta&s&cally  significant   8   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  9. 9. Sins  of  unethical  repor&ng  on  research*    1)  Accentua&ng  the  posi&ve  and  ignoring  the  nega&ve    2)  Generalizing  from  anecdotes    3)  Not  asking  for  the  evidence    4)  Wrong  or  insufficient  interpreta&on  of  numbers    5)  Ignoring  conflicts  of  interest    6)  Offering  misleading  results    7)  Ignoring  the  holis&c  picture  and  failing  to  recognize              the  conclusions  and  weaknesses  of  scien&fic  studies    8)  Ignoring  the  certainty  of  uncertainty  in  science  *Ethics  in  science  journalism,  George  Claassen,  University  of  Stellenbosch   9   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  10. 10. Example  —  Poor  sample  size   10   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  11. 11. Example  —  Exaggerated  claims   11   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  12. 12. Components  of  a  minimally  good    methodology  explana&on    Invita&ons  were  emailed  to  4,847  people  on  a  list  of  ASBPE  members  and  non-­‐member   business-­‐to-­‐business  (B2B)  editors.  Between  November  2  and  December  10,  2009,     338  people  answered  the  survey  for  a  response  rate  of  7%.    An  ini&al  screening  ques&on  eliminated  48  people  who  said  they  were  not  staff  editors   or  writers  for  a  business-­‐to-­‐business  print  or  digital  publica&ons.  Among  the  290  who   indicated  they  were  B2B  staff  editors  or  writers,  17  stopped  answering  the  survey  aaer   the  ini&al  screening  ques&on.  The  results  are  based  on  the  273  remaining  B2B  staff   editors  and  writers.    The  95%  confidence  level  was  used  in  this  study.  Results  labeled  as  sta&s&cally   significant  have  a  5%  or  less  chance  that  they  could  be  aMributed  to  sampling  error   (drawing  a  oddball  sample).    The  survey  ques&ons  received  between  203  and  269  non-­‐missing  responses  (e.g.   people  who  answered  “don’t  know”  or  lea  a  ques&on  blank  are  missing  responses).    Percentages  from  these  ques&ons  have  a  margin  of  sampling  error  from  ±6.0%  (for   ques&ons  with  269  responses)  to  ±6.9%  (for  ques&ons  with  203  responses).  So  using  a   ±7%  margin  of  sampling  error  is  safe  for  all  ques&ons.    This  means  that  sampling  error  should  cause  no  more  than  a  7%  difference  between   the  results  in  our  study  and  the  true  value  in  the  universe  in  95%  of  samples.    For  example,  our  finding  that  81%  of  B2B  editors  received  a  day  or  less  of  digital   training  in  the  past  year  has  a  95%  chance  of  falling  in  the  interval  between  74%  (81%   –  7%)  and  88%  (81%  +  7%)  in  the  universe  from  which  the  sample  was  drawn.   12   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  13. 13. From  Reynolds  Na&onal  Center  for  Business  Journalism  research  Business  Journalists  study    The  informaIon  contained  in  this  report  is  based  on  473  in-­‐depth  interviews  with   business  journalists  throughout  the  United  States.  Interviewing  was  conducted   between  April  19  and  May  6,  2010,  by  professional  interviewers  at  BRC’s  state-­‐of-­‐ the-­‐art  Computer-­‐Assisted  Telephone  Interviewing  (CATI)  facility  in  Phoenix.   Interviewing  was  conducted  under  the  direct  supervision  of  BRC  supervisory   personnel,  who  randomly  monitor  interviews  as  they  are  conducted  and  who   validate  completed  interviews.  Prior  to  beginning  the  interviews,  all  interviewers   were  trained  in  all  aspects  of  this  job,  including  protocols  for  open-­‐ended  quesIons   and  sampling  techniques.    When  reviewing  the  results  from  this  survey,  it  should  be  kept  in  mind  that  all   surveys  are  subject  to  sampling  error.  Sampling  error,  simply  stated,  is  the   difference  between  the  results  obtained  from  a  sample  and  those  that  would  be   obtained  by  surveying  the  enIre  universe  under  consideraIon.  The  overall   sampling  error  for  this  survey  is  approximately  +/-­‐4.6  percent  at  a  95  percent   confidence  interval.   13   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  14. 14. What  editors  must  do  (at  minimum)    Is  sample  random  and  adequate?    Is  confidence  level  adequate?    Is  margin  of  error  is  adequate?    Is  P  value  is  adequate?    Ask  a  3rd  party,  independent  sta&s&cian/researcher     to  review  methodology.    With  your  ar&cle  about  the  research,  include  as  much  info   about  the  methodology  to  give  reader  minimum  way  to   judge  validity.      AND  explain  the  above  to  your  readers.   14   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  15. 15.   Follow  ASBPE  guidelines.    Don’t  publish  ar&cles  about  research   with  poor  methodology.    Don’t  misinform  your  readers.    Don’t  sin.   15   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  16. 16. Addi&onal  resources    American  Cancer  Society   hMp://pubs.acs.org/userimages/ContentEditor/1218054468605/ethics.pdf    Australian  Press  Council     www.rjionline.org/media-­‐accountability/code-­‐of-­‐ethics/      apc-­‐guidelines-­‐for-­‐repor&ng    American  Educa&onal  Research  Associa&on     www.aera.net/uploadedFiles/Publica&ons/Journals/Educa&onal_Researcher/        3506/12ERv35n6_Standard4Report%20.pdf    Blackwell  Publishing     hMp://www.blackwellpublishing.com/publica&onethics    Survey  SoJware  Success,  Jeffrey  Henning,  Founder  &  VP,  Strategy,  Vovici   hMp://www.vovici.com/_assets/pdf/ebook/SurveySoawareSuccess.pdf    Web  Center  for  Social  Research  Methods   hMp://www.socialresearchmethods.net   16   Covering  Research:  Obliga&on  of  the  Editor              Robin  Sherman  Editorial  &  Design  Services   5/25/11  
  17. 17. Repor&ng  on  Research    The  Ethical  Obliga&ons  of  the  Editor   Robin  Sherman,  Editorial  &  Design  Services   RobinSherman@bellsouth.net  

×