Recommended
Recommended
More Related Content
Similar to DVC meetup
Similar to DVC meetup (20)
Recently uploaded
Recently uploaded (20)
DVC meetup
- 3. Enter Data Version Control
- 7. Index ● Explicit data and process dependencies ● Data and model caching ● Visualize metrics across model and data versions ● “One click” pipeline reproducibility ● 🍻 🍕
- 8. Explicit data and process dependencies raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics
- 9. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics Explicit data and process dependencies
- 10. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics $ dvc add data/raw Explicit data and process dependencies
- 11. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics Explicit data and process dependencies
- 12. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics $ dvc run --file prepare_data.dvc --deps prepare_data.ipynb --deps data/raw --outs data/prepared papermill prepare_data.ipynb prepare_data_out.ipynb Explicit data and process dependencies
- 13. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics Explicit data and process dependencies
- 14. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics $ dvc run --file extract_features.dvc --deps extract_features.ipynb --deps data/prepared --outs data/features papermill extract_features.ipynb extract_features_out.ipynb Explicit data and process dependencies
- 15. raw Prepare data prepared train prepared test Extract features features train features test Select model model Test model metrics Explicit data and process dependencies
- 16. raw Prepare data prepared train prepared test Extract features features train features test Select model model Test model metrics $ dvc run --file select_model.dvc --deps select_model.ipynb --deps model.py --deps data/features --outs model papermill select_model.ipynb select_model_out.ipynb Explicit data and process dependencies
- 17. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics Explicit data and process dependencies
- 18. raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics $ dvc run --file test_model.dvc --deps test_model.ipynb --deps data/features --deps model --deps model.py --metrics test_metrics.json papermill test_model.ipynb test_model_out.ipynb Explicit data and process dependencies
- 19. Explicit data and process dependencies $ dvc pipeline show --ascii select_model.dvc $ dvc pipeline show --ascii --outs select_model.dvc
- 20. Data and model caching raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics
- 21. Data and model caching raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics CHANGE HERE
- 22. Data and model caching raw Prepare data prepared train prepared test Extract features features train features test Select model Test model model metrics CHANGE HERE $ dvc repro test_model.dvc
- 23. Data and model caching
- 24. $ dvc metrics show -T Visualize metrics across model and data versions
- 25. “One click” pipeline reproducibility $ git clone git@gitlab.com:adsmurai/training/dvc-meetup.git $ git pull --tags $ dvc pull --all-tags --all-branches $ dvc checkout $ dvc repro test_model.dvc
- 27. Thank you