As the commercial world accelerates investment into AI and Machine Learning one theme continually appears. Models are being built, but they are not being used. Teams of Data Scientists around the world are training versatile models but due to managerial, logistical and infrastructural problems, these models are not making it to production. To watch the full presentation with visual and audio click here: https://info.cnvrg.io/ml-models-to-production In this webinar, Solutions Architect Aaron Schneider will diagnose the problem and identify the symptoms. He’ll explain how reproducibility, scalability and collaboration can increase the gap between research and production. The webinar will examine best practices for building a machine learning pipeline that enables quick iteration, deployment and CI/CD to ensure that your company is deploying and maintaining the best services for you customers and clients. Key takeaways: - The common issues that block deployment and increase time to production - How different stakeholders can resolve key issues - How to accelerate from research to production Tools that can make productionizing models easy - Leveraging Kubernetes and container-based architecture for faster deployment Watch the full presentation here: https://info.cnvrg.io/ml-models-to-production

2. whoami
● Solutions Architect @ cnvrg.io
● = built by data scientists, for data scientists to help teams:
○ Get from data to models to production in the most efficient and fast way
○ Bridge science and engineering
○ Automate MLOps
○ Help teams streamline every element of their pipelines
Aaron
Schneider
aaron@cnvrg.io
LinkedIn: azschneider

3. def agenda(30 mins):
● Understanding the problems and symptoms
● Discussing the root causes
● Best practices for closing the gap
● Tools to streamline the production process

4. Deployment
ML.overview()
Data Processing Training MonitorDeployment
Deployment
Deployment

5. production.statistics()
https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/
Algorithmia: 2020 state of enterprise machine learning
87%
of ML projects
don’t make it to
production
55%
of companies
haven’t deployed a
single model

6. production.statistics()
https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/
Algorithmia: 2020 state of enterprise machine learning
50%
of models take 8-90 days
before production
13%
take 91-365 days
Investment in ML is
increasing

7. Model != in_production
● Good work being done by good data scientists
● Functional and accurate models being made
● But not being used
● If being used, often take a long time before implemented
● Undermines the whole process as ML not being used or no longer
accurate
● Enterprise problem impacting business results
● Need to accelerate time from research into production

8. production_issue(team_friction)
● Multiple teams:
○ Data Science Team (Development)
○ Engineering Team (DevOps)
● Data Science can build the model but engineering can’t (won’t?) put it
into production
● Causes:
○ Miscommunication
○ Developed in Python but implement in C/Java
○ Mismanagement

9. production_issue(iteration)
● You can’t reach desired accuracy through one grid search. Have to
continually improve the model
● Can get stuck in the grind towards the perfect model
● Management wants higher accuracy before production
● Like any software/agile method: Start with MVP
● Build a model and get it to production, THEN monitor and iterate
● Already puts you leaps and bounds ahead of competitors
Accuracy =
62%
Version 2
Accuracy =
60%
Version 1
Accuracy =
64%
Version 3

10. production_issue(containerizing)
● How do we even implement a model? What technology do we use?
● Model developed separately from product, how do we align?
● Containerization is a logical answer
● Tough to implement properly
● Links back to first issue we discussed

11. production_issue(scaling)
● Demand isn’t static
● Need complicated infrastructure to support the service you are
delivering
● Must be able to scale without interruption to service
● Kubernetes is great, but hard
● Need whole different set of skills to build the architecture for
autoscaling with k8

12. production_issue(executive_buy_in)
● Might have the best DS team in the world
● But if higher-ups push back, all a waste of time
● Need the management and executive on board with the vision
● Might be excited by the flashiness of ML/AI/Big Data but need to take
risks and believe in the DS teams capability

13. cnvrg.demo()

14. webinar.summary()
● There are many issues that keep models from production
● Very rarely are they DS issues, usually team friction and engineering
complexities
● It is the responsibility of the DevOps and Management teams to build a
structure that can minimize time to production
● This is a real metric that can have real business consequences, both
positively and negatively
● Low time to production = $$$
● High time to production = -$$$

16. Thanks!
https://cnvrg.io
info@cnvrg.io
+972506600186