Continuous Deployment for Deep Learning

1. Continuous Deployment for Deep Learning — Nick Pentreath Principal Engineer @MLnick

2. About IBM Developer / © 2019 IBM Corporation – @MLnick on Twitter & Github – Principal Engineer, IBM CODAIT (Center for Open-Source Data & AI Technologies) – Machine Learning & AI – Apache Spark committer & PMC – Author of Machine Learning with Spark – Various conferences & meetups 2

3. CODAIT Improving the Enterprise AI Lifecycle in Open Source Center for Open Source Data & AI Technologies IBM Developer / © 2019 IBM Corporation 3 CODAIT aims to make AI solutions dramatically easier to create, deploy, and manage in the enterprise. We contribute to and advocate for the open-source technologies that are foundational to IBM’s AI offerings. 30+ open-source developers!

4. Agenda IBM Developer / © 2019 IBM Corporation – Overview of Continuous Integration & Deployment – The Machine Learning Workflow – How is CI/CD for ML Different & Challenges – Model Asset Exchange – Conclusion 4

21. Deep learning pipeline IBM Developer / © 2019 IBM Corporation 21 beagle: 0.82 basset: 0.09 bluetick: 0.07 ... Input image Image pre-processing Prediction Decode image Resize Normalization Convert types / format Inference Post-processing [0.2, 0.3, … ] (label, prob) Sort Label map PIL, OpenCV, tf.image, … Custom Python * Logos trademarks of their respective projects

22. Pipelines, not Models – Deploying (and testing) just the model part of the workflow is not enough – Entire pipeline must be taken into account • Data transforms • Feature extraction & pre-processing • DL / ML model • Prediction transformation – Even ETL is part of the pipeline! – Pipelines in frameworks • scikit-learn • Spark ML pipelines • TensorFlow Transform • pipeliner (R) IBM Developer / © 2019 IBM Corporation 22

24. Source of changes IBM Developer / © 2019 IBM Corporation 24 Build Test Merge Deploy Dependencies Changes come from: • Our code • Internal dependencies • 3rd party dependencies • Data • Model • Time Data Model

26. Models IBM Developer / © 2019 IBM Corporation – Size of models – Resource requirements – Hardware • CPU, GPU, TPU, Mobile, Edge – Need to manage and bridge many different languages, frameworks – Formats – State of the art is changing very rapidly * Logos trademarks of their respective projects 26

28. Monitoring Performance Business Monitoring Software Traditional software monitoring Latency, throughput, resource usage, etc Model performance metrics Traditional ML evaluation measures (accuracy, prediction error, AUC etc) Business metrics Impact of predictions on business outcomes • Additional revenue - e.g. uplift from recommender • Cost savings – e.g. value of fraud prevented • Metrics implicitly influencing these – e.g. user engagement IBM Developer / © 2019 IBM Corporation

29. Feedback Data Transform TrainDeploy Feedback Adapt An intelligent system must automatically learn from & adapt to the world around it Continual learning Retraining, online learning, reinforcement learning Feedback loops Explicit: models create or directly influence their own training data Implicit: predictions influence behavior in longer-term or indirect ways Humans in the loop IBM Developer / © 2019 IBM Corporation

33. What is MAX? 33 - One place for state-of-art open source deep learning models - Wide variety of domains - Tested code and IP - Free and open source - Both trainable and trained versions IBM Developer / © 2019 IBM Corporation

34. Deployable asset on MAX IBM Developer / © 2019 IBM Corporation 34 Data ExpertiseComputeModel REST API specificationPre Trained ModelI/O processing Deployable Asset on Model Asset Exchange MetadataInferenceSwagger Specification

39. Trainable asset on MAX IBM Developer / © 2019 IBM Corporation 39 Data ExpertiseComputeModel REST API specificationPre Trained ModelI/O processing Deployable Asset on Model Asset Exchange Standardized scripts & wrappers

40. MAX and CI/CD IBM Developer / © 2019 IBM Corporation 40 – TravisCI for GitHub repositories – Jenkins for more complex workflows • Deploying long-running instances to Kubernetes • Periodic jobs • Trainable model testing (using Watson Machine Learning service)

41. Conclusion – ML Pipelines, not just models – Need to take into account • Data • Models • Variation over time • Fairness, explainability, transparency – Containers / Kube? • KubeFlow, MLFlow • Tekton IBM Developer / © 2019 IBM Corporation 41

42. Thank you IBM Developer / © 2019 IBM Corporation Sign up for IBM Cloud and try Watson Studio: https://ibm.biz/BdznGk codait.org twitter.com/MLnick github.com/MLnick developer.ibm.com 42 Model Asset Exchange: https://ibm.biz/model-exchange

Continuous Deployment for Deep Learning

You are reading a preview.

Check these out next

Continuous Deployment for Deep Learning

More Related Content

Slideshows for you (20)

Similar to Continuous Deployment for Deep Learning (20)

More from Databricks (20)

Recently uploaded (20)