Open Science in Psychology

  • 64 views
Uploaded on

Talk given at the National Science Foundation on issues related to replication and data sharing in the psychological sciences (February 22, 2014)

Talk given at the National Science Foundation on issues related to replication and data sharing in the psychological sciences (February 22, 2014)

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
64
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Replication & Data Sharing: The Publication Perspective Bradford W. Hesse, PhD Chair-Elect,APA Publications & Communications Board American Psychological Association Friday, February 14, 14
  • 2. APA Monitor: Feb 2013* Complex Problem • Bias toward positive findings ➡ “P-Hacking,” with false positives in literature ➡ The “file drawer” problem, with false negatives obscured • Insufficient detail for method replication • Lack of incentives to replicate • Disincentives for data sharing * Winerman, Lea. (2013). Interesting results: Can they be replicated? Monitor on Psychology, 44, 38-41. Friday, February 14, 14
  • 3. Task Force on Data Sharing* Necessary for verification * Data sharing memo,April 4, 2012 Restriction in purpose: No reason for sharing other than “verification” Burdensome, costs to be borne by replicating “professional” Friday, February 14, 14
  • 4. Trends Toward Open Science* * e.g., Hesse, B.W., Croyle, R.T., & Buetow, K. H. (2011). Cyberinfrastructure and the biomedical sciences.Am J Prev Med, 40(5 Suppl 2), S97-102. •Computer infrastructure •NSF, NIH requirements •Open Science mandates •Emergence of “Big” problems •Emergence of “Big Data” •Team Science to solve problems Friday, February 14, 14
  • 5. 2 Sides of Same Coin* Data Sharing • Enables replication • Promotes aggregation for knowledge synthesis, hypothesis generation, programmatic decisions, and generalizability testing • Opens data for analysis with more powerful analytic techniques than possible originally • Encourages a culture of openness in science Replication • Technical replication ➡ Method validation ➡ Verification • Conceptual replication ➡ Theory building ➡ Generalizability ➡ Model testing • Culture change ➡ Self-correcting ➡ Science as public trust * Replication and Data Sharing (RADS) Task Force, October 2013 Friday, February 14, 14
  • 6. * Replication and Data Sharing (RADS) Task Force, October 2013 Proposed Policy Revision* Friday, February 14, 14
  • 7. Consider:Article of the Future* * http://www.articleofthefuture.com/about Direct Access to Data Interactive Content Links to Author community (e.g., ORCID) Ability to publish ePub only material Friday, February 14, 14
  • 8. APA’s Cyber-infrastructure* • Credible and ubiquitous portal to psychological sciences • Expanded scope promotes team science, with bidirectional links between data services (e.g., PubMed & PsycINFO) • Direct access to bibliographic tools (e.g., *.pdf’s, cited references, reference software exportation) elevates scientific productivity, reduces friction to good scholarship * Advised by Electronic Resources Advisory Committee (ERAC) Friday, February 14, 14
  • 9. Replication ePubs in all Journals* • Subject to same, rigorous review as print pubs • Not considered against impact factor • May be highlighted by editor as important contribution to field • May be open to online comments at some point in future * Proposal by Council of Editors online only Friday, February 14, 14
  • 10. APA’s Open Science • Free and open to the public • Standard, finely nuanced descriptions of study’s rationale, method, results, and interpretation • Data collaboration ➡ data underlying analysis available ➡ data generators retain authorship rights • Technical + Public abstracts • Open comment discussions Archives of Scientific Psychology* * Courtesy Harris Cooper (Duke University) and GaryVandenBos (APA) Friday, February 14, 14
  • 11. Ten Prescriptions* Improve the transparency of research 1. Require research reports to include complete accounts of method, analyses and results 2. Require data sharing 3. Clarify rights and responsibilities of parties to data- sharing agreements 4. Construct a prospective database of IRB- approved research *Cooper, Harris & VandenBos, Gary. (2013). Ten Prescriptions To Foster Better Social Science in the Internet Environment. Final Report on NIH Grant. Duke University. Durham, NC. Friday, February 14, 14
  • 12. Change the reward structure of research 5. Value replication 6. Train researchers in record keeping and data management 7. Use approaches to evaluations of research (and researchers) that promote transparent science Ten Prescriptions* *Cooper, Harris & VandenBos, Gary. (2013). Ten Prescriptions To Foster Better Social Science in the Internet Environment. Final Report on NIH Grant. Duke University. Durham, NC. Friday, February 14, 14
  • 13. Improve interaction with the global audience 8. Take responsibility for open access publication costs 9. Provide the means for interaction between researchers and an audience of non-researchers 10.Make special efforts to include members of the international community in training Ten Prescriptions* *Cooper, Harris & VandenBos, Gary. (2013). Ten Prescriptions To Foster Better Social Science in the Internet Environment. Final Report on NIH Grant. Duke University. Durham, NC. Friday, February 14, 14
  • 14. Task Force Recommendations Coordinate with other stakeholders in psychological science Work with APA’s Board of Scientific Affairs to engage Office of Human Subjects Protection & State IRB’s Draft publishable document for Replication and Data Sharing Guidelines (similar to Journal Article Reporting Standards) Friday, February 14, 14
  • 15. Reinventing Discovery The real question is: How do we use the methods and discoveries of the psychological sciences to solve problems of replicability, generativity, & integration across all sciences? Friday, February 14, 14