Download as PDF, PPTX
Uploaded byGetInData
Model serving made easy using Kedro pipelines - Mariusz Strzelecki, GetInData
This document discusses using Kedro to streamline the model serving process in machine learning projects by integrating it with Seldon and MLflow. It emphasizes the advantages of combining training and serving in one project, enabling faster iteration and consistency in environments, although it notes some performance overhead. The document highlights Kedro's role in improving reproducibility and bridging the gap between data science and software development.
More Related Content
Similar to Model serving made easy using Kedro pipelines - Mariusz Strzelecki, GetInData
Model serving made easy using Kedro pipelines - Mariusz Strzelecki, GetInData
- 1. Model serving madeeasy using Kedro pipelines Mariusz Strzelecki ML Engineer, GetInData www.ml.dssconf.pl
- 2. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Let’s serve a model! ● Mlflow ● Seldon ● … ● Or just a microservice in python ● https://docs.seldon.io/projects/seldon-core/en/v1.6.0/examples/iris.html
- 3. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Easy, right? ● “Real” models have way more parameters ● “Real” models require some transformations: ○ Values scaling ○ Encoding ● “Real” models should validate input ● “Real” models may depend on data from Feature Store ● “Real” models may require post-processing
- 4. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Custom code already supported ● MLflow offers pyfunc
- 5. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Custom code already supported ● Seldon provides “Python Components”
- 6. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Custom code already supported ● or even multi-microservice inference graph
- 7. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Awesome, right? ● Right! ● But: ○ You keep every model in 2 subprojects: training and serving ○ Either you write tailored code for every model as Python class ○ … or you keep reusable components as multiple separate Docker containers ● Basically: a lot of work to do
- 8. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. What if I told you, that: ● There is a way to keep both: training and serving in one project ● You can test your serving (aka. “inference”) part without leaving Jupyter ● It glues two existing frameworks
- 9.
- 10. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Why Kedro? ● There is no standard on how ML projects should look like ● …, but, there are known good practices! ● Reproducibility is the key ● Fills the gap between Data Science and software development
- 11. © Copyright. Allrights reserved. Not to be reproduced without prior written consent.
- 12. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Building blocks of Kedro project Icons made by dDara and Freepik from Flaticon Node Pipeline Data catalog
- 13. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Node
- 14. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Pipeline
- 15. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Data Catalog
- 16. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Building blocks of Kedro project Icons made by dDara and Freepik from Flaticon Node Pipeline Data catalog
- 17.
- 18. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Kedro Pipelines Data preparation Data science Load dataset 1 Load dataset 2 Join datasets Normalize data Split normalized data Train model Evaluate model Train Test Model Inference Load input Decorate input Normalize data Evaluate model Format output
- 19. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. The code
- 20. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Ups and downs + No longer need to keep 2 separate projects + Super fast iterating on the model + Same docker image enforces identical python version and all libraries - Not as fast as it could be (~11ms of overhead) - Need to install compatible seldon-core package
- 21. © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Summary ● Kedro + Seldon + Mlflow make a useful serving toolchain ● Kedro proved (again) to be all-in-one environment for ML ● Great method for fast iterating, but not an option if every millisecond counts
- 22.