This document outlines the machine learning lifecycle and architecture including data preparation, model building and training, deployment, and DevOps processes. It shows how raw data moves from data lakes to optimized data lakes and clean data stores. Models are built, tracked, and experimented on before being containerized and deployed to production. ML DevOps uses CI/CD to deploy models to AI, BI, and other platforms. Key technologies include Databricks, Azure ML, and Data Lake Analytics.

1. ACI
Data Source
Prepare Data Build & Train Deploy ML DevOps &
Productionisation
Data Movement Raw Data Lake
LifecyleArchitectureTechnologies
Transform
Data Cleansing &
Transformation
Optimized Data Lake
& Clean Data Stores
Build & Train Track & Experiment ML Model Containerisation
Dev/Test
Deploy
Prod
Deploy
Deploy
ML DevOps
(CI/CD)
AI
BI
Data Layer
Databricks Runtime AML VMs Data Lake Analytics
Azure ML SDK
DBFSData Lake
Platform
Databricks
Workspace
AML Workspace
Extract
Extract
Extract
Load Use Track Choose Register
SDK
Compute
Machine Learning Lifecycle & Architecture
created by Ryan Chen
https://www.linkedin.com/in/ryan-chen-71a89054/