Making machine learning model deployment boring - Big Data Expo 2019
•
0 likes•58 views
The free lunch for machine learning is over. Organizations are quickly ramping up their abilities to automate and professionalize their machine learning processes and infrastructure. As a consequence organizational goals, processes and requirements put an increasing burden on teams to put machine learning models in production. We believe much of this burden relates to engineering issues, which with proper abstractions can be greatly reduced for product teams. In this presentation we will talk about the organizational context of ING and the design our Machine Learning Platform. In the first part we will sketch some organizational context and the requirements it brings. Next, we will picture the kind of use cases and user journey we have in mind. Finally, we will present how these considerations led the platform design we are currently deploying.
More Related Content
Similar to Making machine learning model deployment boring - Big Data Expo 2019
Similar to Making machine learning model deployment boring - Big Data Expo 2019 (20)
More from webwinkelvakdag
More from webwinkelvakdag (20)
Recently uploaded
Recently uploaded (20)
Making machine learning model deployment boring - Big Data Expo 2019
- 1. Making machine learning model deployment boring Thomas Kluiters, ING Ruurtjan Pul, BigData Republic
- 2. ING wil softwaregeorienteerde bank worden (steeds meer producten zijn digitaal) en data driven zijn en dus ML gebruiken
- 3. ING wil softwaregeorienteerde bank worden (steeds meer producten zijn digitaal) en data driven zijn en dus ML gebruiken
- 4. Machine learning flips the equation Classical programming Rules Data Answers
- 5. Machine learning flips the equation Classical programming Rules Data Answers Machine learning Answers Data Rules
- 6. Machine learning helps many companies ● Match candidates and vacancies ● Recommend customer treatments to call center agents ● Pick the right time for airplane maintenance
- 7. Machine learning helps many companies ● Match candidates and vacancies ● Recommend customer treatments to call center agents ● Pick the right time for airplane maintenance
- 8. Machine learning helps many companies ● Match candidates and vacancies ● Recommend customer treatments to call center agents ● Pick the right time for airplane maintenance
- 9. Machine learning requires many capabilities Statistics Algorithms Math Scripting Programming CI/CD Feature engineering Data plumbing Monitoring Stability Reliability Infrastructure Business value Compliance and Risk Integration Process Domain knowledge
- 10. Machine learning requires many capabilities Statistics Algorithms Math Scripting Programming CI/CD Feature engineering Data plumbing Monitoring Stability Reliability Infrastructure Business value Compliance and Risk Integration Process Domain knowledge
- 11. Machine learning requires many capabilities Statistics Algorithms Math Scripting Programming CI/CD Data plumbing Monitoring Stability Reliability Infrastructure Programming CI/CD Feature engineering Data plumbing Monitoring Stability Reliability Infrastructure Business value Compliance and Risk Integration Process Domain knowledge
- 12. Machine learning requires many capabilities Statistics Algorithms Math Scripting Programming CI/CD Feature engineering Data plumbing Monitoring Stability Reliability Infrastructure Statistics Algorithms Math Scripting Monitoring Stability Reliability Infrastructure Business value Compliance and Risk Integration Process Domain knowledge
- 13. Machine learning requires many capabilities Statistics Algorithms Math Scripting Programming CI/CD Feature engineering Data plumbing Monitoring Stability Reliability Infrastructure Statistics Algorithms Math Scripting Programming CI/CD Data plumbingBusiness value Compliance and Risk Integration Process Domain knowledge
- 14. Expertise oriented teams do not work Data lab Product team Data engineering Operations
- 15. Expertise oriented teams do not work Data lab Product team Data engineering Operations
- 16. Expertise oriented teams do not work Data lab Product team Data engineering Operations
- 17. Expertise oriented teams do not work Data lab Product team Data engineering Operations
- 18. Expertise oriented teams do not work Data lab Product team Data engineering Operations
- 19. Expertise oriented teams do not work Data lab Product team Data engineering Operations
- 20. Full stack teams don’t scale Feature team
- 21. Full stack teams don’t scale
- 22. Full stack teams don’t scale
- 23. Centralized teams should handle cross-cutting concerns Feature teams Centralized team
- 24. The machine learning platform
- 25. The machine learning platform ● The platform is a service built by ING engineers, for ING engineers and data scientists ● The platform is responsible for hosting machine learning models, and orchestrating them
- 26. The machine learning platform
- 27. The machine learning platform ● Orchestrator as an interface between models and outside world ● Kafka for streaming data ● REST API for real-time and batch
- 28. ● Allow teams to fully embrace the power and capabilities of machine learning ● Only software engineers, data scientists and product owners are responsible The machine learning platform
- 29. A journey: the feedback use case ● ING gathers user feedback ● Feedback covers many categories
- 30. ● Automatically categorize user feedback using machine learning ● Establish business goals Inception of the ideaProduct owner
- 31. ● Data scientists iteratively build a model ● Incorporate feedback from product owner ● Configure the model according to platform standards Building the modelData scientist
- 32. ● Data scientist and software engineer agree on configuration of the model ● Software engineer builds interface for consuming the platform ● Data scientist deploys model on pipeline Packaging the modelSoftware engineer & data scientist
- 33. ● Automated pipelines deploy the model on the platform ● Technologies such as Ansible, Docker and Gitlab CI ● No actions are required by the platform team The packaged modelPlatform
- 34. The productionized model ● Once in production, data scientists, software engineers and product owners can monitor their model ● The model can be changed iteratively and quickly deployed again
- 35. The productionized model, monitoring
- 36. The productionized model, versioning ● Models are tracked by their version number ● Using historic data we can track the performance of the models
- 37. Centralized teams should handle cross-cutting concerns ING’s platform team provides machine learning deployment as a service Allowing data scientists to easily and quickly deploy models makes machine learning boring Key takeaways