Reporting on Research: The Ethical Obligations of the Editor

Repor&ng on Research   The Ethical Obliga&ons of the Editor Robin Sherman, Editorial & Design Services RobinSherman@bellsouth.net

Agenda   Ethical cornerstones   Goal of research   A few sta&s&cal basics   Sins of unethical repor&ng on research   Example   Components of a minimally good methodology explana&on   What editors must do   Resources 2 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

From ASBPE’s Guide To Preferred Editorial Prac&ces Sec&on II. Standards for Editorial Opera&ons D. Research In the case of any research, rankings, “best of” or “worst of” awards, buyers’ guides or similar editorial products, a neutral third party, such as a consulIng ﬁrm, may be used to help avoid conﬂicts of interest. In any editorial content, a clear and complete discussion of the methodology, including methodological and analy&cal limita&ons, should be published to allow the reader to make informed judgments about the value of the content. Sec&on III. Graphics and Photography B. Charts and Graphics Informa&on graphics should include an explana&on of research methodology and give the source. This applies as well to graphics obtained from third parIes. Why do we need these guidelines? 3 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Ethical cornerstones   Accuracy   AMribu&on   Context   Proper emphasis   Transparency 4 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Research goal 1) To whom do you want 1) The theore@cal popula@on to generalize? 2) What popula&on 2) The study popula@on can you get access to? 3) How can you get access 3) The sampling frame to them? 4) Who is in your study? 4) The sample, sub-‐sample 5 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

A few sta&s&cal basics   Survey Responses needed for 95% Confidence with ±5% Margin of Error Sample size considera&ons   Popula&on Size Responses Needed 10 10   Margin of error 5% is the commonly accepted choice. The   100 80 margin of error is the amount of error that   200 132 you can tolerate.   300 169   Confidence level   400 197 Typical choices are 90%, 95%, 99%. Measures the probability that you can   500 218 project survey findings onto the enIre   600 235 universe. s.   700 249   Popula&on size   800 260 How many people are there to choose your random sample from? The sample size   900 270 doesnt change much for populaIons larger   1,000 278 than 20,000.   2,000 323   5,000 357   10,000 370   20,000 377   100,000 383   1,000,000 384 6 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

The Raosoa Sample Size Calculator hMp://raosoa.com/samplesize.html 7 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

A few sta&s&cal basics   P (probability) value Less than .05 usually considered sta&s&cally signiﬁcant 8 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Sins of unethical repor&ng on research*   1) Accentua&ng the posi&ve and ignoring the nega&ve   2) Generalizing from anecdotes   3) Not asking for the evidence   4) Wrong or insufficient interpreta&on of numbers   5) Ignoring conflicts of interest   6) Offering misleading results   7) Ignoring the holis&c picture and failing to recognize the conclusions and weaknesses of scien&fic studies   8) Ignoring the certainty of uncertainty in science *Ethics in science journalism, George Claassen, University of Stellenbosch 9 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Example — Poor sample size 10 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Example — Exaggerated claims 11 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Components of a minimally good methodology explana&on   Invita&ons were emailed to 4,847 people on a list of ASBPE members and non-‐member business-‐to-‐business (B2B) editors. Between November 2 and December 10, 2009, 338 people answered the survey for a response rate of 7%.   An ini&al screening ques&on eliminated 48 people who said they were not staff editors or writers for a business-‐to-‐business print or digital publica&ons. Among the 290 who indicated they were B2B staff editors or writers, 17 stopped answering the survey aaer the ini&al screening ques&on. The results are based on the 273 remaining B2B staff editors and writers.   The 95% confidence level was used in this study. Results labeled as sta&s&cally significant have a 5% or less chance that they could be aMributed to sampling error (drawing a oddball sample).   The survey ques&ons received between 203 and 269 non-‐missing responses (e.g. people who answered “don’t know” or lea a ques&on blank are missing responses).   Percentages from these ques&ons have a margin of sampling error from ±6.0% (for ques&ons with 269 responses) to ±6.9% (for ques&ons with 203 responses). So using a ±7% margin of sampling error is safe for all ques&ons.   This means that sampling error should cause no more than a 7% difference between the results in our study and the true value in the universe in 95% of samples.   For example, our finding that 81% of B2B editors received a day or less of digital training in the past year has a 95% chance of falling in the interval between 74% (81% – 7%) and 88% (81% + 7%) in the universe from which the sample was drawn. 12 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

From Reynolds Na&onal Center for Business Journalism research Business Journalists study   The informaIon contained in this report is based on 473 in-‐depth interviews with business journalists throughout the United States. Interviewing was conducted between April 19 and May 6, 2010, by professional interviewers at BRC’s state-‐of-‐ the-‐art Computer-‐Assisted Telephone Interviewing (CATI) facility in Phoenix. Interviewing was conducted under the direct supervision of BRC supervisory personnel, who randomly monitor interviews as they are conducted and who validate completed interviews. Prior to beginning the interviews, all interviewers were trained in all aspects of this job, including protocols for open-‐ended quesIons and sampling techniques.   When reviewing the results from this survey, it should be kept in mind that all surveys are subject to sampling error. Sampling error, simply stated, is the diﬀerence between the results obtained from a sample and those that would be obtained by surveying the enIre universe under consideraIon. The overall sampling error for this survey is approximately +/-‐4.6 percent at a 95 percent conﬁdence interval. 13 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

What editors must do (at minimum)   Is sample random and adequate?   Is conﬁdence level adequate?   Is margin of error is adequate?   Is P value is adequate?   Ask a 3rd party, independent sta&s&cian/researcher to review methodology.   With your ar&cle about the research, include as much info about the methodology to give reader minimum way to judge validity.   AND explain the above to your readers. 14 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

  Follow ASBPE guidelines.   Don’t publish ar&cles about research with poor methodology.   Don’t misinform your readers.   Don’t sin. 15 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Addi&onal resources   American Cancer Society hMp://pubs.acs.org/userimages/ContentEditor/1218054468605/ethics.pdf   Australian Press Council www.rjionline.org/media-‐accountability/code-‐of-‐ethics/ apc-‐guidelines-‐for-‐repor&ng   American Educa&onal Research Associa&on www.aera.net/uploadedFiles/Publica&ons/Journals/Educa&onal_Researcher/ 3506/12ERv35n6_Standard4Report%20.pdf   Blackwell Publishing hMp://www.blackwellpublishing.com/publica&onethics   Survey SoJware Success, Jeﬀrey Henning, Founder & VP, Strategy, Vovici hMp://www.vovici.com/_assets/pdf/ebook/SurveySoawareSuccess.pdf   Web Center for Social Research Methods hMp://www.socialresearchmethods.net 16 Covering Research: Obliga&on of the Editor Robin Sherman Editorial & Design Services 5/25/11

Repor&ng on Research   The Ethical Obliga&ons of the Editor Robin Sherman, Editorial & Design Services RobinSherman@bellsouth.net

http://www.asbpe.org	371
http://vps4557.inmotionhosting.com	5
http://www.asbpe.org.	2
url_unknown	1

Reporting on Research: The Ethical Obligations of the Editor

by asbpe

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 379

Statistics

Reporting on Research: The Ethical Obligations of the Editor Presentation Transcript