Kamil A. Kaczmarek presentation from 2018 Data Science Summit in Warsaw

1. steppy: lightweight, open-source,
Python library for fast
and reproducible experimentation
Kamil A. Kaczmarek
kamil@neptune.ml

2. Fast and reproducible
experimentation

3. Agenda
● Target audience :)
● Technical debt
● Steppy
● Transition to steppy

4. Target audience

5. Target audience
Data Scientist
DL/ML
Researcher
run a lot of experiments
works on her research project

6. Target audience
https://imgs.xkcd.com/comics/machine_learning.png (retrieved Jun 7th, 2018)

7. Technical debt

8. Technical debt

9. Technical debt
Model
development
ML
Infrastructure
● Do you have (simple) reference model (think baseline)?
● What is the impact of each hyperparameter on model
performance?
● Are training procedures reproducible?
● Is your end-to-end pipeline integrated?
● Can you “roll back” to the previous version?

10. Steppy

11. Steppy
Step
Transformer
● Execution wrapper over the Transformer
● Represents computational step
● Performs operation on (intermediate) data

12. Steppy minimal example (1/5)

13. Steppy minimal example (2/5)

14. Steppy minimal example (3/5)

15. Steppy minimal example (4/5)

16. Steppy minimal example (5/5)

17. Steppy, model ensemble example
https://github.com/minerva-ml/steppy-examples/blob/master/tutorials/3-adapter_advanced.ipynb

18. Steppy, text processing example
https://github.com/minerva-ml/open-solution-toxic-comments

19. Transition to steppy

20. Transition to steppy
From scratch
By example
● pip3 install steppy
● https://steppy.readthedocs.io
● https://github.com/minerva-ml/steppy-examples
● https://github.com/minerva-ml/open-solution-h
ome-credit
● https://github.com/minerva-ml/open-solution-
mapping-challenge

21. Summary

22. Fast and reproducible experimentation
● Simple programming model with Step and Transformer
● No operation outside Step
● Steppy handles model training and testing
● Steppy saves intermediate results
● Steppy saves transformers (especially trained model)
● Steppy visualizes entire graph of computations
● Steppy let’s you exchange particular steps easily
Why do you need steppy?

23. Thank You
Feel free to contact me:
Your
photo
kamil@neptune.ml

24. Transition to steppy
From scratch
By example
● pip3 install steppy
● https://steppy.readthedocs.io
● https://github.com/minerva-ml/steppy-examples
● https://github.com/minerva-ml/open-solution-h
ome-credit
● https://github.com/minerva-ml/open-solution-
mapping-challenge