This presentation was given by David Mellor of the Center for Open Science during the joint NISO-ICSTI event held on October 26, 2016 on the topic of enabling innovation in researcher workflow and scholarly communication.

1. An Open Science Framework for Managing
and Sharing Research Workflows
David Mellor
Journal and Funder Ini1ates
h5ps://cos.io | h5ps://osf.io | @OSFramework

2. Mission:
Increase the openness, integrity,
and reproducibility of scien1fic
research.

3. Strategy:
• Evidence to encourage change.
• Training and incentives to embrace
change.
• Technology to enable change.

4. Infrastructure
Metascience
Community

6. Root of the problem:
Incen1ves for individual success
are focused on geHng it
published, not geHng it right
Nosek, Spies, & Motyl, 2012

7. Problems in research and publishing:
Flexibility in analysis
Selec1ve repor1ng
Ignoring nulls
Lack of replica1on
Sterling, 1959; Cohen, 1962; Lykken, 1968; Tukey, 1969; Greenwald, 1975; Meehl, 1978;
Rosenthal, 1979

8. Technology to enable change

10. h5p://osf.io/
free, open source

11. Collaboration
Documentation
Archiving

13. Put data, materials, and code on the
OSF

14. Manage access and permissions

15. Version
Control

17. Registration

19. Merges Public-
Private
Workflows

21. Incentives to embrace change

22. Persistent
Citable
Identifiers

24. File downloads
Incentives
for
Openness

25. File downloads
Forks
Visits

26. Signals: Making Behaviors Visible
Promotes Adop1on

27. 0%
10%
20%
30%
1st Half 2012 2nd Half 2012 1st Half 2013 2nd Half 2013 1st Half 2014 2nd Half 2014 1st Half 2015
PercentageofArticleswithOpenData
Independent Repository, Badge
Independent Repository, No Badge
PSCI Supplement, Badge
PSCI Supplement, No Badge
Independent Website, Badge
Independent Website, No Badge
Personal website, No Badge
Data Availability in Psychological Science

28. The Prereg Challenge
$1,000,000 in prizes for researchers to publish the results of
their preregistered work.
Makes dis1nc1on between exploratory
and confirmatory research more clear.

29. Exploratory research:
• Finds unexpected trends
• Pushes knowledge into new areas
• Results in a testable hypothesis

30. Confirmatory research:
• Puts a hypothesis to the test
• Does not allow data to influence the hypothesis
• Results are held to the highest standard of rigor

31. https://cos.io/prereg

32. Research ques1ons
Data collec1on methods
Variables
Sta1s1cal tests
Outliers
https://cos.io/prereg

33. Connecting the workflow
is critical to enabling change

34. Connects Services
Researchers Use

36. OpenSesame

37. OpenSesame

38. API Docs
https://api.osf.io/v2/docs/

39. Let Experts be Experts

40. OSF
Application
Framework
Workflow
Authentication
File Storage
File Rendering
Meta-database
Integrations
Search
SHARE Data
osf.io

41. OSF
Application
Framework
Workflow
Authentication
File Storage
File Rendering
Meta-database
Integrations
Search
SHARE Data
osf.io
journals
registries
preprint servers
grants management
curation, annotation
peer review services

44. Let Experts be Experts

45. Substan1ve experts
shouldn’t have to deal with…
• Workflow integration
• Authentication
• Permissions
• File storage
• File rendering
• Database
• Metadata/Annotations/Commenting
• Persistence
• External service integrations
• Search
• SHARE Data

46. Substan1ve experts
should focus on…
• Defining community standards
• Changing community values and incentives
• Determining specific community requirements
• Doing scholarship

47. Society-branded Preprint Servers

48. OSF for InsJtuJons

49. Thank you!
David Mellor, david@cos.io
Find me online: https://osf.io/qthsf/
Find this presentation:
https://osf.io/6zcw6