Recommended
Recommended
More Related Content
What's hot
What's hot (18)
Similar to NAS Data Science Roundtable: Training as a Pathway to Improve Reproducibility
Similar to NAS Data Science Roundtable: Training as a Pathway to Improve Reproducibility (20)
Recently uploaded
Recently uploaded (20)
NAS Data Science Roundtable: Training as a Pathway to Improve Reproducibility
- 1. Training as a Pathway to Improve Reproducibility Tracy K. Teal, PhD The Carpentries, Executive Director
- 2. There is increasing awareness around reproducibility in research
- 3. With increasing awareness, there is increasing interest in working reproducibly
- 4. Working reproducibly requires new skills and new ways of working. It’s not an easy shift.
- 5. Researchers need to connect the theory of reproducibility with the practical skills and application.
- 6. Reproducibility for Future You
- 9. People want to learn the skills http://braembl.org.au/news/braembl-community-survey-report-2013
- 10. Current unmet needs Barone L, Williams J and Micklos D. Unmet Needs for Analyzing Biological Big Data: A Survey of 704 NSF Principal Investigators (2017)
- 11. How do we scale training along with data production and reach all communities?
- 12. Developing and providing scalable training 1. Training ‘in the gaps’ 2. Collaborative and open educational resources 3. Building communities of practice
- 13. 1. Training in the Gaps Needs to Be… • Accessible • Approachable • Aligned • Applicable The 4 A’s
- 14. Delivery of the 4 A’s • Integrating into existing courses • Short courses and workshops • MOOCs • Online just-in-time training
- 15. Non-profit organization that develops curriculum, trains instructors and teaches workshops on the skills and perspectives to work effectively and reproducibly with software and data.
- 16. Workshops 2-days, active learning Feedback to learners throughout the workshop Trained instructors Friendly learning environment Skills and perspectives applicable to data workflow and software development best practices.
- 17. Workshop goals • Teach skills • Get people started and introduce them to what’s possible • Build confidence in using these skills • Encourage people to continue learning • Positive learning experience
- 18. Do Carpentries workshops work?
- 20. People like the workshops
- 22. Confidence and self-perception to program change significantly
- 23. Attitudes around reproducible research change
- 24. Increased confidence and change in perspective persists longer term
- 26. Also an impact on use of skills
- 27. 2. Collaborative and open educational resources • Accessible • Approachable • Aligned • Applicable The 4 A’s
- 29. If you want to go fast, go alone. If you want to go far, go together.
Editor's Notes
- When we talk about reproducibility, we’re not just talking about ‘for the greater good’. We’re also talking about reproducibility for ‘future you’ and that is often the most motivating.
- Addmittedly much in this survey is the skills to be able
- Also we see in our workshops. Fill within hours. For instance a recent workshop at the NIH. Had 20 spots. Had a wait list of 84 people.
- How do we make sure that all people have the opportunity to ‘level up’ in their reproducible research practices and not just some people. If this is how we’re saying research must be done, we must make sure there’s equal access to opportunities and resources that make this possible. What can we be doing now?
- Accessible: at a time and location when people can go, accessible for all learners, meeting accessibility requirments, available, findable as a resource Approachable: meeting people where they are, friendly learning environment, approachable instructors Aligned: aligned with domain, interest and current needs Applicable: immediately applicable to their work
- Lots of great efforts in different areas. Berkeley’s course, Data Camp, online videos
- Software and Data Carpentry are non-profit organizations that – train people in software development and data science skills for more effective research and work and for career development and – build community and local capacity for teaching and sharing these skills and perspectives
- Workshops are 2-day, hands-on, interactive, friendly learning environment (CoC), teaching the foundational skills and perspectives for working with software and data
- Goals of the workshop, aren’t just to teach the skills, but to build self-efficacy and increase confidence and create a positive learning experience. We know we can’t teach everything in two days, but we want to teach the foundational skills and get people started and give them the confidence to continue learning. Many people have had demotivating experience when learning things like coding or computational skills, and we want to change that perspective.
- Not much research on effectiveness of training in data science or computer science
- Since 2014 taught 857 workshops to over 17,000 learners
- Also, they just like the workshop, feeling it was a good use of their time and that they would recommend it to a colleague or friend.
- AnalysesEasier: Using a programming language (like R or Python) can make my analyses easier to reproduce. OvercomeProblem: While working on a programming project, if I get stuck, I can find ways of overcoming the problem. ProgrammingSoftware: I am confident in my ability to make use of programming software to work with data. RawData: Having access to the original, raw data is important to be able to repeat an analysis. SearchOnline: I know how to search for answers to my technical questions online. WriteScript: I can write a small program/script/macro to solve a problem in my own work.
- AnalysesEasier: Using a programming language (like R or Python) can make my analyses easier to reproduce. OvercomeProblem: While working on a programming project, if I get stuck, I can find ways of overcoming the problem. ProgrammingSoftware: I am confident in my ability to make use of programming software to work with data. RawData: Having access to the original, raw data is important to be able to repeat an analysis. SearchOnline: I know how to search for answers to my technical questions online. WriteScript: I can write a small program/script/macro to solve a problem in my own work.
- AnalysesEasier: Using a programming language (like R or Python) can make my analyses easier to reproduce. OvercomeProblem: While working on a programming project, if I get stuck, I can find ways of overcoming the problem. ProgrammingSoftware: I am confident in my ability to make use of programming software to work with data. RawData: Having access to the original, raw data is important to be able to repeat an analysis. SearchOnline: I know how to search for answers to my technical questions online. WriteScript: I can write a small program/script/macro to solve a problem in my own work.
- I have made my analyses more reproducible as a result of completing the workshop. I have received professional recognition for my work as a result of using the tools I learned at the workshop. My research productivity has improved as a result of completing the workshop. I have been motivated to seek more knowledge about the tools I learned at the workshop. I have gained confidence in working with data as a result of completing the workshop. I have improved my coding practices as a result of completing the workshop. I have used skills I learned at the workshop to advance my career.
- Accessible: discoverable and open, modular so that you can use elements of it and remix Approachable: the notes you would need to teach it without having to talk to the person Aligned: aligned with the needs of your learners Applicable:
- This is all building towards a community of practice. A community of learners and instructors and teaching each other. We talk about culture change, but I would talk about community of practice. Setting norms and standards for the community, but also improving practices and building a community of people who support each other in working this way and continuing to learn
- In today’s world of increasing reliance on software and a growing trend towards data-driven work, we think these skills are very important and crucial for helping students reach their full potential in their career. But it’s not only the skills, but buildlng a community – a community of practice that uses these skills and continues to learn and support each other, and a community that teaches and mentors each other. So many people do want to learn these things, but are trying to learn them on their own or don’t have opportunities to learn.There’s a saying ‘if you want go fast, go alone, but if you want to go far go together’ and we’re focused on the going together to create life-long learners and a sustainable community.