open analyses

1. José A. Ruipérez Valiente, PhD (@JoseARuiperez)
UMU – University of Murcia – (jruiperez@um.es)
Senior Researcher
IEEE Member
5.4. Open Analyses
EDUCATION SOCIETY
http://ieee-edusociety.org/
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories

2. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 2
Introduction
•Objective: Allowing easy reproduction of the analytics
conducted by the researchers in a study
• This makes very easy to conduct reproduction studies
• Helpful to verify methods and errors
• This enables easy replication of any research study
• Reusing analytical methods in a different type of study
•Plays well with code tracking tools and platforms like
GitHub or SVN
• History of changes in code representing the evolution of the
research methodology

3. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 3
Reproducible research tools
• There are many statistical tools out there. These are
important factors to take into account:
• Proprietary vs free software: is it available to everyone?
• Use of interoperable data formats
• Programming vs. point-and-click interface
• Flexible library system and active communities
• The recommendation is programming languages that are
free and open. The most extended are R and Python
• Deployment tools like Docker or Ansible for environments
• Ethical discussion and relevance of proprietary software
• Nice video about benefits of reproducible research

4. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 4
Jupyter notebooks and R Markdown
•They combine narrative, code and output
• Exportable or hosted in different formats
• Use Markdown syntax for document design
•Jupyter notebooks were part of IPython
• It has now support to execute other code (e.g. R)
• More info in Jupyter.org, comes with anaconda distribution
• See the MOOC Pivot notebook and others in this gallery
•R Markdown come with R Studio IDE
• Is compiled with knitr to HTML or PDF
• Tutorial R Markdown and R Markdown Cheatsheet
• Lots of of R Markdown examples in this gallery

5. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 5
Jupyter interface

6. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 6
R Markdown interface

7. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 7
Conclusions
• Open design and open data enables the use of open analyses for
full reproducibility
• HTML outputs with code/outputs are the most interoperable and easily
hosted in the web
• Code is only good if it can be understood and replicated
• Write readable code (avoid obfuscation)
• Write comments explaining decisions
• Don’t perform any manual changes outside the code
• In the future, there could be complete transparent publications in
these interactive environments
• Applies in qualitative research too (coding schemes, coded or
disagreement examples, etc)
• Follow guidelines like DRESS protocol or OECD principles

8. EDUCATION SOCIETY
http://ieee-edusociety.org/
Subunit – 5.4 Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories
Slide 8
References
• van der Zee, T., & Reich, J. (2018). Open education science. AERA Open, 4(3), 2332858418787466.
• Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B. E., Bussonnier, M., Frederic, J., . . . Ivanov, P. (2016). Jupyter
Notebooks–A publishing format for reproducible computational workflows. In ELPUB (pp. 87–90). Amsterdam: IOS
Press.
• Somers, J. (2018, April 5). The scientific paper is obsolete. The Atlantic -
https://www.theatlantic.com/science/archive/2018/04/the-scientific-paper-is-obsolete/556676/
• Anderson, C. (2010). Presenting and evaluating qualitative research. American journal of pharmaceutical education,
74(8).

9. EDUCATION SOCIETY
http://ieee-edusociety.org/
José A. Ruipérez Valiente, PhD (@JoseARuiperez)
UMU – University of Murcia – (jruiperez@um.es)
Senior Researcher
IEEE Member
5.4. Open Analyses
Unit 5 – Open Education Science
MOOC – Foundations to Open Education and OERs repositories