Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Study pre-registration: Benefits and considerations

972 views

Published on

SANS 2017

Published in: Science
  • Be the first to comment

  • Be the first to like this

Study pre-registration: Benefits and considerations

  1. 1. STUDY PRE-REGISTRATION Benefits and considerations
  2. 2. THE PROBLEM Science isn’t perfect.
  3. 3. 53eig.ht/HackingScience
  4. 4. The first principle is that you must not fool yourself and you are the easiest person to fool. Richard P. Feynman
  5. 5. Different types of bias http://www.nature.com/articles/s41562-016-0021
  6. 6. THE SOLUTION Pre-registration
  7. 7. What is a preregistration? It’s the introduction and methods section of your future paper.
  8. 8. What should be included in a preregistration? • Hypotheses • Predictions • Models • Dependent variables • ROIs • Confounds • Exclusion criteria • Feature definition (“functional connectivity defined as…”) • Analysis plan • Statistical techniques (algorithms) • Multiple comparison correction • Parameters
  9. 9. Types of preregistration 1. Reviewed Pre-Registration (RPR aka “Registered Report”) •Registration is submitted to a journal and reviewed by experts 1. Unreviewed Pre-Registration (UPR) •Pre-registration is deposited in a public or private repository
  10. 10. Reviewed Pre-Registration https://www.elsevier.com/editors-update/story/peer-review/cortexs-registered-reports
  11. 11. Reviewed Pre-Registration • Advances in Methodologies and Practices in Psychological Science • AIMS Neuroscience • Animal Behavior and Cognition • Attention, Perception, and Psychophysics • Behavioral Neuroscience • Cognition and Emotion • Cognitive Research: Principles and Implications • Comprehensive Results in Social Psychology • Cortex • Drug and Alcohol Dependence • European Journal of Neuroscience • Experimental Psychology • Health Psychology Bulletin • Human Movement Science • Infancy • International Journal of Psychophysiology • Journal of Business and Psychology • Journal of Cognitive Enhancement • Journal of European Psychology Students • Journal of Experimental Political Science • Journal of Media Psychology • Journal of Personnel Psychology • Journal of Research in Personality • Judgment and Decision Making • Management and Organization Review • Memory • Nature Human Behaviour • NFS Journal • Nicotine &Tobacco Research • Perspectives on Psychological Science • Royal Society Open Science • Stress and Health • The Leadership Quarterly • Work, Aging and Retirement
  12. 12. Reviewed Pre-Registration http://dx.doi.org/10.1016/j.cortex.2016.01.006
  13. 13. Unreviewed Pre-Registration Where to submit? https://aspredicted.org/ https://osf.io/
  14. 14. AsPredicted.org Demo
  15. 15. Benefits for science • Reduced publication bias http://dx.doi.org/10.1371/journal.pone.0132382
  16. 16. Benefits for science •Reduced reporting bias •Clear distinction between exploratory and confirmatory research •Prioritizing theory and method http://dx.doi.org/10.1371/journal.pone.0132382
  17. 17. Benefits for the researcher •Pre-analysis review feedback (RPR only) •Guaranteed publication (RPR only) •Clarity among collaborators http://dx.doi.org/10.1371/journal.pone.0132382
  18. 18. Pre registration misconceptions • It’s too much work • It does not allow for exploratory science • It will flood the literature with “boring” null results • My reviewers will scoop me • It forces researchers to stick to their hypotheses instead of listening to the data
  19. 19. Q & A
  20. 20. van ’tVeer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A discussion and suggested template. Journal of experimental social psychology, 67, 2–12. http://dx.doi.org/10.1016/j.jesp.2016.03.004

×