Presented by Daniel Whitenack, Data Scientist and Lead Developer Advocate at Pachyderm

1. Putting the science back in
Data Science
Daniel Whitenack, @dwhitena
Data Scientist and Advocate, @pachydermIO
Chicago Data Science Conference 2017

2. Outline
1. Why do we care about Reproducibility?
2. How can we achieve Reproducibility?
3. Demo with R and Pachyderm
4. Resources
@dwhitena, @datascienceassn, #DSAChicago2017

3. Why do we care about
Reproducibility?
@dwhitena, @datascienceassn, #DSAChicago2017

4. How can we achieve
Reproducibility?
(at scale)
@dwhitena, @datascienceassn, #DSAChicago2017

5. Demo
@dwhitena, @datascienceassn, #DSAChicago2017

6. iris.csv
1.3,1.4,...
@dwhitena, @datascienceassn, #DSAChicago2017

7. iris.csv
1.3,1.4,...
@dwhitena, @datascienceassn, #DSAChicago2017

8. iris.csv
1.3,1.4,...
train.R
model.rda
model.txt
@dwhitena, @datascienceassn, #DSAChicago2017

9. iris.csv
1.3,1.4,...
train.R
infer.R
model.rda
model.txt
@dwhitena, @datascienceassn, #DSAChicago2017

10. iris.csv
1.3,1.4,...
train.R
infer.R
1.csv
1.3,1.4,...
1
setosa
model.rda
model.txt
@dwhitena, @datascienceassn, #DSAChicago2017

11. iris.csv
1.3,1.4,...
train.R
infer.R
1.csv
1.3,1.4,...
1
setosamodel.rda
model.txt
@dwhitena, @datascienceassn, #DSAChicago2017

12. … enter Pachyderm
An open source, distributed processing and data versioning
framework built on containers.
@dwhitena, @datascienceassn, #DSAChicago2017

13. Pachyderm
training
model
model
attributes
1.csv
inference
1
Running train.Riris.csv
inference
Running infer.R
model.rda
model.txt
@dwhitena, @datascienceassn, #DSAChicago2017

14. Pachyderm
training
model
model
attributes
1.csv
inference
1
Running train.Riris.csv
Inference 1
model.rda
model.txt
Inference 2
Inference N
@dwhitena, @datascienceassn, #DSAChicago2017

15. Pachyderm
training
model
model
attributes inference
inference
@dwhitena, @datascienceassn, #DSAChicago2017

16. Pachyderm
training
model
model
attributes inference
inference
plots
plots
@dwhitena, @datascienceassn, #DSAChicago2017

17. Pachyderm
training
model
model
attributes inference
inference
plots
plots
raw_data
training
@dwhitena, @datascienceassn, #DSAChicago2017

18. training
model
model
attributes inference
inference
plots
plots
raw_data
training
raw_attr
attributes
#!/bin/bash
@dwhitena, @datascienceassn, #DSAChicago2017

19. Pachyderm
training
model
model
attributes inference
inference
plots
plots
raw_data
training
raw_attr
attributes
attributes
attributes
inference
training
@dwhitena, @datascienceassn, #DSAChicago2017

20. Conclusion/Resources
● Run the code/pipeline
● Join the Pachyderm Slack channel
● Check out the Pachyderm docs
● Slack/tweet me @dwhitena
● Read a related article
@dwhitena, @datascienceassn, #DSAChicago2017