Downloaded 67 times
Uploaded byDatabricks
Productionizing Machine Learning Pipelines with Databricks and Azure ML
The document discusses the process of deploying machine learning applications using Databricks and Azure ML, highlighting the challenges and solutions in moving from development to production. It includes a tutorial overview and key learnings regarding machine learning templates, infrastructure code, and model management techniques. The document also features a live tutorial on deep learning image classification utilizing a dataset of 25,000 images stored in Azure blob storage.
More Related Content
Similar to Productionizing Machine Learning Pipelines with Databricks and Azure ML
![[AI] ML Operationalization with Microsoft Azure](https://cdn.slidesharecdn.com/ss_thumbnails/wds-mlops-trainer-kyle-akepanidtaworn-v03-190924094657-thumbnail.jpg?width=640&height=640&fit=bounds)
Productionizing Machine Learning Pipelines with Databricks and Azure ML
- 2. Productionizing Machine Learning Pipelineswith Databricks & Azure ML Trace Smith & Amirhessam Tahmassebi Data Scientist - ExxonMobil
- 3.
- 4. Machine Learning Templates ▪Deployment of Machine Learning Applications ▪ Significant amount of time moving from dev to production ▪ Resources and experience to design and implement ▪ Developing Reproducible Frameworks ▪ Quickly jumpstart and accelerate Data Science projects into production ▪ Series of code repositories with production-ready code templates ▪ Developed around previous use-cases ▪ Segment into infrastructure code and business logic ▪ Data Scientist more focused on developing models ▪ Template Example ▪ Integrating Databricks with Azure Machine Learning and Azure DevOps for end-to-end model deployment Model ManagementModel TrainingData Ingestion Azure Machine Learning Azure Pipelines
- 5. Tutorial Overview source: https://www.kaggle.com/c/dogs-vs-cats ▪Deep Learning: Computer Vision ▪ Image Classification – Convolution Neural Network ▪ Libraries: Tensorflow & PyTorch ▪ Dataset: Cats and Dog ▪ Open source dataset ▪ Reference: https://www.microsoft.com/en-us/download/ ▪ Total Images: 25,000 ▪ Images stored in Azure Blob Storage ▪ Mounted to Databricks Workspace Source: https://cs231n.github.io/convolutional-networks/
- 6. Key Learnings ▪ CustomContainerization ▪ Databricks-connect ▪ Deep Learning (CNN) ▪ Parallelized Gridsearch ▪ Model Management w/ MLflow ▪ AzureML Workspace ▪ AzureML Pipelines ▪ Azure Model Registry ▪ Custom inference script ▪ Webservice Deployment Azure Machine LearningMachine Learning • Azure DevOps • Code Quality Checks ▪ Automated Unit Testing ▪ Azure Pipelines ▪ Continuous deployment Deployment
- 8.
- 9.
- 10. Feedback Your feedback isimportant to us. Don’t forget to rate and review the sessions.