SlideShare a Scribd company logo
Continuous Deployment
for Deep Learning
—
Nick Pentreath
Principal Engineer
@MLnick
About
IBM Developer / © 2019 IBM Corporation
– @MLnick on Twitter & Github
– Principal Engineer, IBM CODAIT (Center for
Open-Source Data & AI Technologies)
– Machine Learning & AI
– Apache Spark committer & PMC
– Author of Machine Learning with Spark
– Various conferences & meetups
2
CODAIT
Improving the Enterprise AI Lifecycle in Open Source
Center for Open Source
Data & AI Technologies
IBM Developer / © 2019 IBM Corporation 3
CODAIT aims to make AI solutions dramatically
easier to create, deploy, and manage in the
enterprise.
We contribute to and advocate for the open-source
technologies that are foundational to IBM’s AI
offerings.
30+ open-source developers!
Agenda
IBM Developer / © 2019 IBM Corporation
– Overview of Continuous Integration &
Deployment
– The Machine Learning Workflow
– How is CI/CD for ML Different & Challenges
– Model Asset Exchange
– Conclusion
4
Continuous Integration
& Deployment
IBM Developer / © 2019 IBM Corporation 5
Testing
IBM Developer / © 2019 IBM Corporation 6
App
Data Store
Public Front-end Services Back-end Services
Service
Remote
Service
Remote
Service
Testing
IBM Developer / © 2019 IBM Corporation 7
App
Data Store
Public Front-end Services Back-end Services
Service
Remote
Service
Remote
Service
Unit testing
Testing
IBM Developer / © 2019 IBM Corporation 8
App
Data Store
Public Front-end Services Back-end Services
Service
Remote
Service
Remote
Service
Integration testing
Testing
IBM Developer / © 2019 IBM Corporation 9
App
Data Store
Public Front-end Services Back-end Services
Service
Remote
Service
Remote
Service
Post-deploy / Acceptance testing
Development Process
IBM Developer / © 2019 IBM Corporation 10
Build Test Merge Deploy
Dependencies
Starting Point
IBM Developer / © 2019 IBM Corporation 11
Build Test Merge
Dependencies
Manual
Automated
Deploy
Continuous Integration
IBM Developer / © 2019 IBM Corporation 12
Build Test Merge
Dependencies
Manual
Automated
Deploy
Continuous Deployment
IBM Developer / © 2019 IBM Corporation 13
Build Test Merge
Dependencies
Manual
Automated
Deploy
Source of changes
IBM Developer / © 2019 IBM Corporation 14
Build Test Merge Deploy
Dependencies
Changes come from:
• Our code
• Internal dependencies
• 3rd party dependencies
The Machine Learning
Workflow
IBM Developer / © 2019 IBM Corporation 15
Perception
IBM Developer / © 2019 IBM Corporation 16
In reality the
workflow spans teams …
IBM Developer / © 2019 IBM Corporation 17
… and tools …
IBM Developer / © 2019 IBM Corporation 18
… and is a small (but critical!)
piece of the puzzle
*Source: Hidden Technical Debt in Machine Learning Systems
IBM Developer / © 2019 IBM Corporation 19
What is a “model”?
IBM Developer / © 2019 IBM Corporation 20
Deep learning pipeline
IBM Developer / © 2019 IBM Corporation 21
beagle: 0.82
basset: 0.09
bluetick: 0.07
...
Input image Image pre-processing Prediction
Decode image
Resize
Normalization
Convert types / format
Inference Post-processing
[0.2, 0.3, … ]
(label, prob)
Sort
Label map
PIL, OpenCV, tf.image,
…
Custom
Python
* Logos trademarks of their respective projects
Pipelines, not Models
– Deploying (and testing) just the model
part of the workflow is not enough
– Entire pipeline must be taken into
account
• Data transforms
• Feature extraction & pre-processing
• DL / ML model
• Prediction transformation
– Even ETL is part of the pipeline!
– Pipelines in frameworks
• scikit-learn
• Spark ML pipelines
• TensorFlow Transform
• pipeliner (R)
IBM Developer / © 2019 IBM Corporation 22
Continuous Integration
for Machine Learning
IBM Developer / © 2019 IBM Corporation 23
Source of changes
IBM Developer / © 2019 IBM Corporation 24
Build Test Merge Deploy
Dependencies
Changes come from:
• Our code
• Internal dependencies
• 3rd party dependencies
• Data
• Model
• Time
Data Model
Data
IBM Developer / © 2019 IBM Corporation
– Data types
• Images, video, audio, raw text,
unstructured
– Size
– Schemas
* Logos trademarks of their respective projects
25
Models
IBM Developer / © 2019 IBM Corporation
– Size of models
– Resource requirements
– Hardware
• CPU, GPU, TPU, Mobile, Edge
– Need to manage and bridge many
different languages, frameworks
– Formats
– State of the art is changing very
rapidly
* Logos trademarks of their respective projects
26
Monitoring & Feedback
over Time
IBM Developer / © 2019 IBM Corporation 27
Monitoring
Performance Business
Monitoring
Software
Traditional software monitoring
Latency, throughput, resource usage, etc
Model performance metrics
Traditional ML evaluation measures
(accuracy, prediction error, AUC etc)
Business metrics
Impact of predictions on business
outcomes
• Additional revenue - e.g. uplift from recommender
• Cost savings – e.g. value of fraud prevented
• Metrics implicitly influencing these – e.g. user
engagement
IBM Developer / © 2019 IBM Corporation
Feedback
Data
Transform
TrainDeploy
Feedback
Adapt
An intelligent system must automatically
learn from & adapt to the world around it
Continual learning
Retraining, online learning,
reinforcement learning
Feedback loops
Explicit: models create or directly
influence their own training data
Implicit: predictions influence behavior
in longer-term or indirect ways
Humans in the loop
IBM Developer / © 2019 IBM Corporation
AI Fairness,
Transparency &
Security
30IBM Developer / © 2019 IBM Corporation
AI Fairness,
Transparency &
Security
31IBM Developer / © 2019 IBM Corporation
https://github.com/IBM/AIX360/
https://github.com/IBM/AIF360
https://github.com/IBM/adversarial-robustness-toolbox
32
Model
Asset
eXchange
(MAX)
ibm.biz/model-exchange
IBM Developer / © 2019 IBM Corporation
What is
MAX?
33
- One place for state-of-art open
source deep learning models
- Wide variety of domains
- Tested code and IP
- Free and open source
- Both trainable and trained versions
IBM Developer / © 2019 IBM Corporation
Deployable asset on MAX
IBM Developer / © 2019 IBM Corporation 34
Data ExpertiseComputeModel
REST API specificationPre Trained ModelI/O processing
Deployable Asset on Model Asset
Exchange
MetadataInferenceSwagger
Specification
Deployable asset on MAX
IBM Developer / © 2019 IBM Corporation 35
MetadataInferenceSwagger
Specification
Deployable asset on MAX
IBM Developer / © 2019 IBM Corporation 36
Hosted resources
IBM Developer / © 2019 IBM Corporation 37
Model
Service
Web App
Kubernetes
Trainable asset on MAX
IBM Developer / © 2019 IBM Corporation 38
Data ExpertiseComputeModel
REST API specificationPre Trained ModelI/O processing
Deployable Asset on Model Asset
Exchange
Trainable asset on MAX
IBM Developer / © 2019 IBM Corporation 39
Data ExpertiseComputeModel
REST API specificationPre Trained ModelI/O processing
Deployable Asset on Model Asset
Exchange
Standardized scripts & wrappers
MAX and CI/CD
IBM Developer / © 2019 IBM Corporation 40
– TravisCI for GitHub repositories
– Jenkins for more complex workflows
• Deploying long-running instances to
Kubernetes
• Periodic jobs
• Trainable model testing (using Watson
Machine Learning service)
Conclusion
– ML Pipelines, not just models
– Need to take into account
• Data
• Models
• Variation over time
• Fairness, explainability,
transparency
– Containers / Kube?
• KubeFlow, MLFlow
• Tekton
IBM Developer / © 2019 IBM Corporation 41
Thank you
IBM Developer / © 2019 IBM Corporation
Sign up for IBM Cloud and try Watson Studio: https://ibm.biz/BdznGk
codait.org
twitter.com/MLnick
github.com/MLnick
developer.ibm.com
42
Model Asset Exchange:
https://ibm.biz/model-exchange
IBM Developer / © 2019 IBM Corporation 43

More Related Content

What's hot

Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud
Vertex AI - Unified ML Platform for the entire AI workflow on Google CloudVertex AI - Unified ML Platform for the entire AI workflow on Google Cloud
Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud
Márton Kodok
 
Serverless machine learning operations
Serverless machine learning operationsServerless machine learning operations
Serverless machine learning operations
Stepan Pushkarev
 
Metaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at NetflixMetaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at Netflix
Bill Liu
 
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Sri Ambati
 
Productionizing Spark ML Pipelines with the Portable Format for Analytics
Productionizing Spark ML Pipelines with the Portable Format for AnalyticsProductionizing Spark ML Pipelines with the Portable Format for Analytics
Productionizing Spark ML Pipelines with the Portable Format for Analytics
Nick Pentreath
 
Machine Learning In Production
Machine Learning In ProductionMachine Learning In Production
Machine Learning In Production
Samir Bessalah
 
Monitoring AI with AI
Monitoring AI with AIMonitoring AI with AI
Monitoring AI with AI
Stepan Pushkarev
 
Grokking Techtalk #40: AWS’s philosophy on designing MLOps platform
Grokking Techtalk #40: AWS’s philosophy on designing MLOps platformGrokking Techtalk #40: AWS’s philosophy on designing MLOps platform
Grokking Techtalk #40: AWS’s philosophy on designing MLOps platform
Grokking VN
 
Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...
Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...
Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...
Databricks
 
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
PAPIs.io
 
Seamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflowSeamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflow
Databricks
 
Machine learning model to production
Machine learning model to productionMachine learning model to production
Machine learning model to production
Georg Heiler
 
Data Science Salon: A Journey of Deploying a Data Science Engine to Production
Data Science Salon: A Journey of Deploying a Data Science Engine to ProductionData Science Salon: A Journey of Deploying a Data Science Engine to Production
Data Science Salon: A Journey of Deploying a Data Science Engine to Production
Formulatedby
 
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
MLOps - Build pipelines with Tensor Flow Extended & KubeflowMLOps - Build pipelines with Tensor Flow Extended & Kubeflow
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
Jan Kirenz
 
ML Infra for Netflix Recommendations - AI NEXTCon talk
ML Infra for Netflix Recommendations - AI NEXTCon talkML Infra for Netflix Recommendations - AI NEXTCon talk
ML Infra for Netflix Recommendations - AI NEXTCon talk
Faisal Siddiqi
 
High Performance Transfer Learning for Classifying Intent of Sales Engagement...
High Performance Transfer Learning for Classifying Intent of Sales Engagement...High Performance Transfer Learning for Classifying Intent of Sales Engagement...
High Performance Transfer Learning for Classifying Intent of Sales Engagement...
Databricks
 
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sri Ambati
 
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Sri Ambati
 
How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...
How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...
How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...
Gustav Lundström
 
TensorFlow 16: Building a Data Science Platform
TensorFlow 16: Building a Data Science Platform TensorFlow 16: Building a Data Science Platform
TensorFlow 16: Building a Data Science Platform
Seldon
 

What's hot (20)

Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud
Vertex AI - Unified ML Platform for the entire AI workflow on Google CloudVertex AI - Unified ML Platform for the entire AI workflow on Google Cloud
Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud
 
Serverless machine learning operations
Serverless machine learning operationsServerless machine learning operations
Serverless machine learning operations
 
Metaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at NetflixMetaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at Netflix
 
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
 
Productionizing Spark ML Pipelines with the Portable Format for Analytics
Productionizing Spark ML Pipelines with the Portable Format for AnalyticsProductionizing Spark ML Pipelines with the Portable Format for Analytics
Productionizing Spark ML Pipelines with the Portable Format for Analytics
 
Machine Learning In Production
Machine Learning In ProductionMachine Learning In Production
Machine Learning In Production
 
Monitoring AI with AI
Monitoring AI with AIMonitoring AI with AI
Monitoring AI with AI
 
Grokking Techtalk #40: AWS’s philosophy on designing MLOps platform
Grokking Techtalk #40: AWS’s philosophy on designing MLOps platformGrokking Techtalk #40: AWS’s philosophy on designing MLOps platform
Grokking Techtalk #40: AWS’s philosophy on designing MLOps platform
 
Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...
Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...
Model Parallelism in Spark ML Cross-Validation with Nick Pentreath and Bryan ...
 
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
 
Seamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflowSeamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflow
 
Machine learning model to production
Machine learning model to productionMachine learning model to production
Machine learning model to production
 
Data Science Salon: A Journey of Deploying a Data Science Engine to Production
Data Science Salon: A Journey of Deploying a Data Science Engine to ProductionData Science Salon: A Journey of Deploying a Data Science Engine to Production
Data Science Salon: A Journey of Deploying a Data Science Engine to Production
 
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
MLOps - Build pipelines with Tensor Flow Extended & KubeflowMLOps - Build pipelines with Tensor Flow Extended & Kubeflow
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
 
ML Infra for Netflix Recommendations - AI NEXTCon talk
ML Infra for Netflix Recommendations - AI NEXTCon talkML Infra for Netflix Recommendations - AI NEXTCon talk
ML Infra for Netflix Recommendations - AI NEXTCon talk
 
High Performance Transfer Learning for Classifying Intent of Sales Engagement...
High Performance Transfer Learning for Classifying Intent of Sales Engagement...High Performance Transfer Learning for Classifying Intent of Sales Engagement...
High Performance Transfer Learning for Classifying Intent of Sales Engagement...
 
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
 
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
 
How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...
How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...
How to combine Db2 on Z, IBM Db2 Analytics Accelerator and IBM Machine Learni...
 
TensorFlow 16: Building a Data Science Platform
TensorFlow 16: Building a Data Science Platform TensorFlow 16: Building a Data Science Platform
TensorFlow 16: Building a Data Science Platform
 

Similar to Continuous Deployment for Deep Learning

IBM Cloud Private and IBM Power Systems: Overview and Real-World Scenarios
IBM Cloud Private and IBM Power Systems: Overview and Real-World ScenariosIBM Cloud Private and IBM Power Systems: Overview and Real-World Scenarios
IBM Cloud Private and IBM Power Systems: Overview and Real-World Scenarios
Joe Cropper
 
Deploying End-to-End Deep Learning Pipelines with ONNX
Deploying End-to-End Deep Learning Pipelines with ONNXDeploying End-to-End Deep Learning Pipelines with ONNX
Deploying End-to-End Deep Learning Pipelines with ONNX
Databricks
 
From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...
From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...
From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...
Luciano Resende
 
Scaling up deep learning by scaling down
Scaling up deep learning by scaling downScaling up deep learning by scaling down
Scaling up deep learning by scaling down
Nick Pentreath
 
Ai pipelines powered by jupyter notebooks
Ai pipelines powered by jupyter notebooksAi pipelines powered by jupyter notebooks
Ai pipelines powered by jupyter notebooks
Luciano Resende
 
IBM Connect AD206 IBM Domino XPages – Embrace, Extend, Integrate
IBM Connect AD206 IBM Domino XPages –  Embrace, Extend, IntegrateIBM Connect AD206 IBM Domino XPages –  Embrace, Extend, Integrate
IBM Connect AD206 IBM Domino XPages – Embrace, Extend, Integrate
Niklas Heidloff
 
Inteligencia artificial, open source e IBM Call for Code
Inteligencia artificial, open source e IBM Call for CodeInteligencia artificial, open source e IBM Call for Code
Inteligencia artificial, open source e IBM Call for Code
Luciano Resende
 
IBM Z for the Digital Enterprise - DevOps for Z
IBM Z for the Digital Enterprise - DevOps for Z IBM Z for the Digital Enterprise - DevOps for Z
IBM Z for the Digital Enterprise - DevOps for Z
DevOps for Enterprise Systems
 
Dev ops for z
Dev ops for z Dev ops for z
Dev ops for z
bamadhu
 
Open Source AI - News and examples
Open Source AI - News and examplesOpen Source AI - News and examples
Open Source AI - News and examples
Luciano Resende
 
End-to-End Deep Learning Deployment with ONNX
End-to-End Deep Learning Deployment with ONNXEnd-to-End Deep Learning Deployment with ONNX
End-to-End Deep Learning Deployment with ONNX
Nick Pentreath
 
150330 verse - uffe sorensen posted
150330   verse - uffe sorensen posted150330   verse - uffe sorensen posted
150330 verse - uffe sorensen posted
Uffe Sorensen
 
MQLight for WebSphere Integration user group June 2014
MQLight for WebSphere Integration user group June 2014MQLight for WebSphere Integration user group June 2014
MQLight for WebSphere Integration user group June 2014
Mark Phillips
 
IBM MQ Light Service for Bluemix
IBM MQ Light Service for BluemixIBM MQ Light Service for Bluemix
IBM MQ Light Service for Bluemix
IBM Systems UKI
 
IBM Enterprise Social Solutions on Bluemix (XPages and Connections)
IBM Enterprise Social Solutions  on Bluemix (XPages and Connections)IBM Enterprise Social Solutions  on Bluemix (XPages and Connections)
IBM Enterprise Social Solutions on Bluemix (XPages and Connections)
Niklas Heidloff
 
Optimize your CI/CD with GitLab and AWS
Optimize your CI/CD with GitLab and AWSOptimize your CI/CD with GitLab and AWS
Optimize your CI/CD with GitLab and AWS
DevOps.com
 
App Modernization
App ModernizationApp Modernization
App Modernization
PT Datacomm Diangraha
 
Ibm business partner connect 2015 long fong yee v1 (read-only)
Ibm business partner connect 2015   long fong yee v1 (read-only)Ibm business partner connect 2015   long fong yee v1 (read-only)
Ibm business partner connect 2015 long fong yee v1 (read-only)
Fong Yee Long
 
Db2 developer ecosystem
Db2 developer ecosystemDb2 developer ecosystem
Db2 developer ecosystem
ModusOptimum
 
How to Balance System Speed and Risk for Multi-Platform Innovation
How to Balance System Speed and Risk for Multi-Platform InnovationHow to Balance System Speed and Risk for Multi-Platform Innovation
How to Balance System Speed and Risk for Multi-Platform Innovation
Claudia Ring
 

Similar to Continuous Deployment for Deep Learning (20)

IBM Cloud Private and IBM Power Systems: Overview and Real-World Scenarios
IBM Cloud Private and IBM Power Systems: Overview and Real-World ScenariosIBM Cloud Private and IBM Power Systems: Overview and Real-World Scenarios
IBM Cloud Private and IBM Power Systems: Overview and Real-World Scenarios
 
Deploying End-to-End Deep Learning Pipelines with ONNX
Deploying End-to-End Deep Learning Pipelines with ONNXDeploying End-to-End Deep Learning Pipelines with ONNX
Deploying End-to-End Deep Learning Pipelines with ONNX
 
From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...
From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...
From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...
 
Scaling up deep learning by scaling down
Scaling up deep learning by scaling downScaling up deep learning by scaling down
Scaling up deep learning by scaling down
 
Ai pipelines powered by jupyter notebooks
Ai pipelines powered by jupyter notebooksAi pipelines powered by jupyter notebooks
Ai pipelines powered by jupyter notebooks
 
IBM Connect AD206 IBM Domino XPages – Embrace, Extend, Integrate
IBM Connect AD206 IBM Domino XPages –  Embrace, Extend, IntegrateIBM Connect AD206 IBM Domino XPages –  Embrace, Extend, Integrate
IBM Connect AD206 IBM Domino XPages – Embrace, Extend, Integrate
 
Inteligencia artificial, open source e IBM Call for Code
Inteligencia artificial, open source e IBM Call for CodeInteligencia artificial, open source e IBM Call for Code
Inteligencia artificial, open source e IBM Call for Code
 
IBM Z for the Digital Enterprise - DevOps for Z
IBM Z for the Digital Enterprise - DevOps for Z IBM Z for the Digital Enterprise - DevOps for Z
IBM Z for the Digital Enterprise - DevOps for Z
 
Dev ops for z
Dev ops for z Dev ops for z
Dev ops for z
 
Open Source AI - News and examples
Open Source AI - News and examplesOpen Source AI - News and examples
Open Source AI - News and examples
 
End-to-End Deep Learning Deployment with ONNX
End-to-End Deep Learning Deployment with ONNXEnd-to-End Deep Learning Deployment with ONNX
End-to-End Deep Learning Deployment with ONNX
 
150330 verse - uffe sorensen posted
150330   verse - uffe sorensen posted150330   verse - uffe sorensen posted
150330 verse - uffe sorensen posted
 
MQLight for WebSphere Integration user group June 2014
MQLight for WebSphere Integration user group June 2014MQLight for WebSphere Integration user group June 2014
MQLight for WebSphere Integration user group June 2014
 
IBM MQ Light Service for Bluemix
IBM MQ Light Service for BluemixIBM MQ Light Service for Bluemix
IBM MQ Light Service for Bluemix
 
IBM Enterprise Social Solutions on Bluemix (XPages and Connections)
IBM Enterprise Social Solutions  on Bluemix (XPages and Connections)IBM Enterprise Social Solutions  on Bluemix (XPages and Connections)
IBM Enterprise Social Solutions on Bluemix (XPages and Connections)
 
Optimize your CI/CD with GitLab and AWS
Optimize your CI/CD with GitLab and AWSOptimize your CI/CD with GitLab and AWS
Optimize your CI/CD with GitLab and AWS
 
App Modernization
App ModernizationApp Modernization
App Modernization
 
Ibm business partner connect 2015 long fong yee v1 (read-only)
Ibm business partner connect 2015   long fong yee v1 (read-only)Ibm business partner connect 2015   long fong yee v1 (read-only)
Ibm business partner connect 2015 long fong yee v1 (read-only)
 
Db2 developer ecosystem
Db2 developer ecosystemDb2 developer ecosystem
Db2 developer ecosystem
 
How to Balance System Speed and Risk for Multi-Platform Innovation
How to Balance System Speed and Risk for Multi-Platform InnovationHow to Balance System Speed and Risk for Multi-Platform Innovation
How to Balance System Speed and Risk for Multi-Platform Innovation
 

More from Databricks

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
Databricks
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1
Databricks
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2
Databricks
 
Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
Databricks
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Databricks
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
Databricks
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
Databricks
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
Databricks
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
Databricks
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch FixThe Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
Databricks
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
Databricks
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Databricks
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
Databricks
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Databricks
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature Aggregations
Databricks
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
Databricks
 
Raven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction QueriesRaven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction Queries
Databricks
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
Databricks
 

More from Databricks (20)

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2
 
Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch FixThe Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature Aggregations
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
 
Raven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction QueriesRaven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction Queries
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
 

Recently uploaded

Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
Roger Valdez
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Fernanda Palhano
 
The Ipsos - AI - Monitor 2024 Report.pdf
The  Ipsos - AI - Monitor 2024 Report.pdfThe  Ipsos - AI - Monitor 2024 Report.pdf
The Ipsos - AI - Monitor 2024 Report.pdf
Social Samosa
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
nyfuhyz
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
wyddcwye1
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
ihavuls
 
DSSML24_tspann_CodelessGenerativeAIPipelines
DSSML24_tspann_CodelessGenerativeAIPipelinesDSSML24_tspann_CodelessGenerativeAIPipelines
DSSML24_tspann_CodelessGenerativeAIPipelines
Timothy Spann
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
Bill641377
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
AlessioFois2
 
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docxDATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
SaffaIbrahim1
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
v7oacc3l
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
Walaa Eldin Moustafa
 
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
xclpvhuk
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
soxrziqu
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
AndrzejJarynowski
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
Social Samosa
 

Recently uploaded (20)

Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
 
The Ipsos - AI - Monitor 2024 Report.pdf
The  Ipsos - AI - Monitor 2024 Report.pdfThe  Ipsos - AI - Monitor 2024 Report.pdf
The Ipsos - AI - Monitor 2024 Report.pdf
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
 
DSSML24_tspann_CodelessGenerativeAIPipelines
DSSML24_tspann_CodelessGenerativeAIPipelinesDSSML24_tspann_CodelessGenerativeAIPipelines
DSSML24_tspann_CodelessGenerativeAIPipelines
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
 
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docxDATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
 
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
 

Continuous Deployment for Deep Learning

  • 1. Continuous Deployment for Deep Learning — Nick Pentreath Principal Engineer @MLnick
  • 2. About IBM Developer / © 2019 IBM Corporation – @MLnick on Twitter & Github – Principal Engineer, IBM CODAIT (Center for Open-Source Data & AI Technologies) – Machine Learning & AI – Apache Spark committer & PMC – Author of Machine Learning with Spark – Various conferences & meetups 2
  • 3. CODAIT Improving the Enterprise AI Lifecycle in Open Source Center for Open Source Data & AI Technologies IBM Developer / © 2019 IBM Corporation 3 CODAIT aims to make AI solutions dramatically easier to create, deploy, and manage in the enterprise. We contribute to and advocate for the open-source technologies that are foundational to IBM’s AI offerings. 30+ open-source developers!
  • 4. Agenda IBM Developer / © 2019 IBM Corporation – Overview of Continuous Integration & Deployment – The Machine Learning Workflow – How is CI/CD for ML Different & Challenges – Model Asset Exchange – Conclusion 4
  • 5. Continuous Integration & Deployment IBM Developer / © 2019 IBM Corporation 5
  • 6. Testing IBM Developer / © 2019 IBM Corporation 6 App Data Store Public Front-end Services Back-end Services Service Remote Service Remote Service
  • 7. Testing IBM Developer / © 2019 IBM Corporation 7 App Data Store Public Front-end Services Back-end Services Service Remote Service Remote Service Unit testing
  • 8. Testing IBM Developer / © 2019 IBM Corporation 8 App Data Store Public Front-end Services Back-end Services Service Remote Service Remote Service Integration testing
  • 9. Testing IBM Developer / © 2019 IBM Corporation 9 App Data Store Public Front-end Services Back-end Services Service Remote Service Remote Service Post-deploy / Acceptance testing
  • 10. Development Process IBM Developer / © 2019 IBM Corporation 10 Build Test Merge Deploy Dependencies
  • 11. Starting Point IBM Developer / © 2019 IBM Corporation 11 Build Test Merge Dependencies Manual Automated Deploy
  • 12. Continuous Integration IBM Developer / © 2019 IBM Corporation 12 Build Test Merge Dependencies Manual Automated Deploy
  • 13. Continuous Deployment IBM Developer / © 2019 IBM Corporation 13 Build Test Merge Dependencies Manual Automated Deploy
  • 14. Source of changes IBM Developer / © 2019 IBM Corporation 14 Build Test Merge Deploy Dependencies Changes come from: • Our code • Internal dependencies • 3rd party dependencies
  • 15. The Machine Learning Workflow IBM Developer / © 2019 IBM Corporation 15
  • 16. Perception IBM Developer / © 2019 IBM Corporation 16
  • 17. In reality the workflow spans teams … IBM Developer / © 2019 IBM Corporation 17
  • 18. … and tools … IBM Developer / © 2019 IBM Corporation 18
  • 19. … and is a small (but critical!) piece of the puzzle *Source: Hidden Technical Debt in Machine Learning Systems IBM Developer / © 2019 IBM Corporation 19
  • 20. What is a “model”? IBM Developer / © 2019 IBM Corporation 20
  • 21. Deep learning pipeline IBM Developer / © 2019 IBM Corporation 21 beagle: 0.82 basset: 0.09 bluetick: 0.07 ... Input image Image pre-processing Prediction Decode image Resize Normalization Convert types / format Inference Post-processing [0.2, 0.3, … ] (label, prob) Sort Label map PIL, OpenCV, tf.image, … Custom Python * Logos trademarks of their respective projects
  • 22. Pipelines, not Models – Deploying (and testing) just the model part of the workflow is not enough – Entire pipeline must be taken into account • Data transforms • Feature extraction & pre-processing • DL / ML model • Prediction transformation – Even ETL is part of the pipeline! – Pipelines in frameworks • scikit-learn • Spark ML pipelines • TensorFlow Transform • pipeliner (R) IBM Developer / © 2019 IBM Corporation 22
  • 23. Continuous Integration for Machine Learning IBM Developer / © 2019 IBM Corporation 23
  • 24. Source of changes IBM Developer / © 2019 IBM Corporation 24 Build Test Merge Deploy Dependencies Changes come from: • Our code • Internal dependencies • 3rd party dependencies • Data • Model • Time Data Model
  • 25. Data IBM Developer / © 2019 IBM Corporation – Data types • Images, video, audio, raw text, unstructured – Size – Schemas * Logos trademarks of their respective projects 25
  • 26. Models IBM Developer / © 2019 IBM Corporation – Size of models – Resource requirements – Hardware • CPU, GPU, TPU, Mobile, Edge – Need to manage and bridge many different languages, frameworks – Formats – State of the art is changing very rapidly * Logos trademarks of their respective projects 26
  • 27. Monitoring & Feedback over Time IBM Developer / © 2019 IBM Corporation 27
  • 28. Monitoring Performance Business Monitoring Software Traditional software monitoring Latency, throughput, resource usage, etc Model performance metrics Traditional ML evaluation measures (accuracy, prediction error, AUC etc) Business metrics Impact of predictions on business outcomes • Additional revenue - e.g. uplift from recommender • Cost savings – e.g. value of fraud prevented • Metrics implicitly influencing these – e.g. user engagement IBM Developer / © 2019 IBM Corporation
  • 29. Feedback Data Transform TrainDeploy Feedback Adapt An intelligent system must automatically learn from & adapt to the world around it Continual learning Retraining, online learning, reinforcement learning Feedback loops Explicit: models create or directly influence their own training data Implicit: predictions influence behavior in longer-term or indirect ways Humans in the loop IBM Developer / © 2019 IBM Corporation
  • 30. AI Fairness, Transparency & Security 30IBM Developer / © 2019 IBM Corporation
  • 31. AI Fairness, Transparency & Security 31IBM Developer / © 2019 IBM Corporation https://github.com/IBM/AIX360/ https://github.com/IBM/AIF360 https://github.com/IBM/adversarial-robustness-toolbox
  • 33. What is MAX? 33 - One place for state-of-art open source deep learning models - Wide variety of domains - Tested code and IP - Free and open source - Both trainable and trained versions IBM Developer / © 2019 IBM Corporation
  • 34. Deployable asset on MAX IBM Developer / © 2019 IBM Corporation 34 Data ExpertiseComputeModel REST API specificationPre Trained ModelI/O processing Deployable Asset on Model Asset Exchange MetadataInferenceSwagger Specification
  • 35. Deployable asset on MAX IBM Developer / © 2019 IBM Corporation 35 MetadataInferenceSwagger Specification
  • 36. Deployable asset on MAX IBM Developer / © 2019 IBM Corporation 36
  • 37. Hosted resources IBM Developer / © 2019 IBM Corporation 37 Model Service Web App Kubernetes
  • 38. Trainable asset on MAX IBM Developer / © 2019 IBM Corporation 38 Data ExpertiseComputeModel REST API specificationPre Trained ModelI/O processing Deployable Asset on Model Asset Exchange
  • 39. Trainable asset on MAX IBM Developer / © 2019 IBM Corporation 39 Data ExpertiseComputeModel REST API specificationPre Trained ModelI/O processing Deployable Asset on Model Asset Exchange Standardized scripts & wrappers
  • 40. MAX and CI/CD IBM Developer / © 2019 IBM Corporation 40 – TravisCI for GitHub repositories – Jenkins for more complex workflows • Deploying long-running instances to Kubernetes • Periodic jobs • Trainable model testing (using Watson Machine Learning service)
  • 41. Conclusion – ML Pipelines, not just models – Need to take into account • Data • Models • Variation over time • Fairness, explainability, transparency – Containers / Kube? • KubeFlow, MLFlow • Tekton IBM Developer / © 2019 IBM Corporation 41
  • 42. Thank you IBM Developer / © 2019 IBM Corporation Sign up for IBM Cloud and try Watson Studio: https://ibm.biz/BdznGk codait.org twitter.com/MLnick github.com/MLnick developer.ibm.com 42 Model Asset Exchange: https://ibm.biz/model-exchange
  • 43. IBM Developer / © 2019 IBM Corporation 43