Chris Fregly, profile picture
Uploaded byChris Fregly
PDF, PPTX4,011 views

KubeFlow + GPU + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark ML + Jupyter

This document provides an overview and agenda for a workshop on end-to-end machine learning pipelines using TFX, Kubeflow, Airflow and MLflow. The agenda covers setting up an environment with Kubernetes, using TensorFlow Extended (TFX) components to build pipelines, ML pipelines with Airflow and Kubeflow, hyperparameter tuning with Kubeflow, and deploying notebooks with Kubernetes. Hands-on exercises are also provided to explore key areas like TensorFlow Data Validation, TensorFlow Transform, TensorFlow Model Analysis and Airflow ML pipelines.

Embed presentation

Download as PDF, PPTX
1 / 105
2 / 105
Most read
3 / 105
4 / 105
5 / 105
6 / 105
7 / 105
8 / 105
9 / 105
10 / 105
11 / 105
12 / 105
13 / 105
14 / 105
15 / 105
16 / 105
17 / 105
18 / 105
19 / 105
20 / 105
21 / 105
22 / 105
23 / 105
24 / 105
25 / 105
26 / 105
27 / 105
28 / 105
29 / 105
30 / 105
31 / 105
32 / 105
33 / 105
34 / 105
35 / 105
36 / 105
37 / 105
38 / 105
39 / 105
40 / 105
41 / 105
42 / 105
43 / 105
44 / 105
45 / 105
46 / 105
47 / 105
48 / 105
49 / 105
50 / 105
51 / 105
52 / 105
53 / 105
54 / 105
55 / 105
56 / 105
57 / 105
58 / 105
59 / 105
60 / 105
61 / 105
62 / 105
63 / 105
64 / 105
65 / 105
66 / 105
67 / 105
68 / 105
69 / 105
70 / 105
71 / 105
72 / 105
73 / 105
74 / 105
75 / 105
76 / 105
77 / 105
78 / 105
79 / 105
80 / 105
81 / 105
82 / 105
83 / 105
84 / 105
85 / 105
86 / 105
87 / 105
88 / 105
89 / 105
90 / 105
91 / 105
92 / 105
93 / 105
94 / 105
95 / 105
96 / 105
97 / 105
98 / 105
99 / 105
100 / 105
101 / 105
102 / 105
103 / 105
104 / 105
105 / 105
End-to-End ML Pipelines TFX + Kubeflow + Airflow + MLflow Chris Fregly Founder @ .
Founder @ PipelineAI Continuous Machine Learning in Production Former Databricks, Netflix Apache Spark Contributor O’Reilly Author High Performance TensorFlow in Production Meetup Organizer Advanced Kubeflow Meetup Who Am I? (@cfregly)
Advanced Kubeflow Meetup (Global+Bay Area, Monthly Events) https://meetup.com/Advanced-Kubeflow-Meetup
Full-Day Workshop https://pipeline.ai @cfregly @PipelineAI
1 OK with Command Line? 2 OK with Python? 3 OK with Linear Algebra? Who are you? 4 OK with Docker? 6 5 OK with Jupyter Notebook?
Recent Poll (July 2019)
4,000 Stars = $6,000,000 Seed $1,500 per GitHub Star?! (Please star the repo ASAP!!) Recent Comment from Popular VC Investor in Silicon Valley
Community Edition https://community.pipeline.ai
After Pushing Your Model to Production, Your Model is… 1 Already Out of Date – Need to Re-Train Biased – Need to Validate Before Pushing Broken – Need to A/B Test in Production Hacked – Need to Train With Data Privacy Slow – Need to Quantize and Speed Up Predictions 2 3 4 5
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
Note #1 of 11 IGNORE WARNINGS & ERRORS (Everything will be OK!)
Note #2 of 11 THERE IS A LOT OF MATERIAL HERE Many opportunities to explore on your own. (Don’t upload sensitive data.)
Note #3 of 11 YOU HAVE YOUR OWN INSTANCE 16 CPU, 104 GB RAM, 200GB SSD (Each with a full Kubernetes Cluster.)
Note #4 of 11 DATASETS Chicago Taxi Dataset (and various others.)
Note #5 of 11 SOME NOTEBOOKS TAKE MINUTES Please be patient. (We are using large datasets)
Note #6 of 11 QUESTIONS? Post questions to Zoom chat or Q&A. (Antje and I will answer soon) Antje >
Note #7 of 11 KUBEFLOW IS NOT A SILVER BULLET There are still gaps in the pipeline. (But gaps are getting smaller)
Note #8 of 11 THIS IS NOT CLOUD DEPENDENT* *Except for 2 small exceptions… (Patches are underway.)
Note #9 of 11 PRIMARILY TENSORFLOW 1.x TF 2.x is not fully supported by TFX (Until Mid-2020.)
Note #10 of 11 SHUTDOWN EACH NOTEBOOK AFTER We are using complex browser voo-doo. (Javascript is a mystery.)
Note #11 of 11 Retrieve 1 Single IP Address Here… <INSERT HERE> (Do not click refresh.)
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
Hands On 00_Explore_Environment
1.1 Kubernetes TensorFlow Extended (TFX) Airflow ML Pipelines 1.0 Environment Overview KubeFlow ML Pipelines 6 Hyper-Parameter Tuning (Katib) Prediction Traffic Router (Istio) 1.2 1.3 1.4 1.6 1.7 MLflow Pipelines1.5
1.1 Kubernetes Kubernetes NFS Ceph Cassandra MySQL Spark Airflow Tensorflow Caffe TF-Serving Flask+Scikit Operating system (Linux, Windows) CPU Memory DiskSSD GPU FPGA ASIC NIC Jupyter GCP AWS Azure On-prem Namespace Quota Logging Monitoring RBAC
Hands On 01_Explore_Kubernetes_Cluster
System 6 System 5System 4 Training At Scale System 3 System 1 Data Ingestion Data Analysis Data Transform Data Validation System 2 Build Model Model Validation Serving Logging Monitoring Roll-out Data Splitting Ad-Hoc Training Why TFX and Why KubeFlow? Improve Training/Serving Consistency Unify Disparate Systems Manage Pipeline Complexity Improve Portability Wrangle Large Datasets Improve Model Quality Manage Versions Composability Distributed Training Configure
1.2 TensorFlow Extended (TFX) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Jenkins?
1.3 Airflow ML Pipelines
1.4 KubeFlow ML Pipelines
1.5 MLflow Experiment Tracking
1.6 Hyper-Parameter Tuning (Katib)
1.7 Prediction Traffic Routing (Istio)
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
2.1 TFX Internals 2.0 TFX Components 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training 2.2 TFX Libraries 2.2 TFX Components
2.1 TFX Internals Driver/Publisher Moves data to/from Metadata Store Executor Runs the Actual Processing Code Metadata Store Artifact, execution, and lineage Info Track inputs & outputs of all components Stores training run including inputs & outputs Analysis, validation, and versioning results Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
2.2 TFX Libraries 2.2.1 TFX Components Use These: TensorFlow Data Validation (TFDV) TensorFlow Transform (TFT) TensorFlow Model Analysis (TFMA) TensorFlow Metadata (TFMD) + ML Metadata (MLMD) 2.2.2 2.2.3 2.2.4 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
2.2.1 TFX Libraries - TFDV TensorFlow Data Validation (TFDV) Find Missing, Redundant & Important Features Identify Features with Unusually-Large Scale `infer_schema()` Generates Schema Describe Feature Ranges Detect Data Drift Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uniformly Distributed Data è ç Non-Uniformly Distributed Data
Hands On 02_TensorFlow_Data_Validation (TFDV)
2.2.2 TFX Libraries - TFT TensorFlow Transform (TFT) Preprocess `tf.Example` data with TensorFlow Useful for data that requires a full pass Normalize all inputs by mean and std dev Create vocabulary of strings è integers over all data Bucketize features based on entire data distribution Outputs a TensorFlow graph Re-used across both training and serving Uses Apache Beam (local mode) for Parallel Analysis Can also use distributed mode `preprocessing_fn(inputs)`: Primary Fn to Implement Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training import tensorflow as tf import tensorflow_transform as tft def preprocessing_fn(inputs): x = inputs['x'] y = inputs['y'] s = inputs['s'] x_centered = x - tft.mean(x) y_normalized = tft.scale_to_0_1(y) s_integerized = tft.compute_and_apply_vocabulary(s) x_centered_times_y_normalized = x_centered * y_normalized return { 'x_centered': x_centered, 'y_normalized': y_normalized, 'x_centered_times_y_normalized':x_centered_times_y_normalized, 's_integerized': s_integerized }
Hands On 03_TensorFlow_Transform (TFT)
Hands On 03a_TensorFlow_Transform_Advanced (TFT)
2.2.3 TFX Libraries - TFMA TensorFlow Model Analysis (TFMA) Analyze Model on Different Slices of Dataset Track Metrics Over Time (“Next Day Eval”) `EvalSavedModel` Contains Slicing Info TFMA Pipeline: Read, Extract, Evaluate, Write ie. Ensure Model Works Fairly Across All Users Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
Hands On 04_TensorFlow_Model_Analysis (TFMA)
2.2.4 TFX Libraries – Metadata TensorFlow Metadata (TFMD) ML Metadata (MLMD) Record and Retrieve Experiment Metadata Artifact, Execution, and Lineage Info Track Inputs / Outputs of All TFX Components Stores Training Run Info Analysis and Validation Results Model Versioning Info Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
2.3 TFX Components ExampleGen StatisticsGen SchemaGen ExampleValidator Evaluator Transform ModelValidator Trainer Model Pusher2.3.92.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.3.6 2.3.7 2.3.8 Slack (!!)2.3.10
2.3.1 ExampleGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Load Training Data Into TFX Pipeline Supports External Data Sources Supports CSV and TFRecord Formats Converts Data to tf.Example Note: TFX Pipelines require tf.Example (?!) Difficult to use non-TF models like XGBoost from tfx.utils.dsl_utils import csv_input from tfx.components.example_gen.csv_example_gen.component import CsvExampleGen examples = csv_input(os.path.join(base_dir, 'data/simple')) example_gen = CsvExampleGen(input_base=examples)
2.3.2 StatisticsGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Generates Statistics on Training Data Global `mean` and `stddev` per input feature Consumes tf.Example instances from tfx import components compute_eval_stats = components.StatisticsGen( input_data=examples_gen.outputs.eval_examples, name='compute-eval-stats' )
2.3.3 SchemaGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Schema Needed by Some TFX Components Data Types, Value Ranges, Optional, Required Consumes Data from StatisticsGen Schema used by TFDV, TFT, TFMA Libraries Uses TFDV Library to infer schema Best effort and basic Human should verify feature { name: "age" value_count { min: 1 max: 1 } type: FLOAT presence { min_fraction: 1 min_count: 1 } } from tfx import components infer_schema = components.SchemaGen( stats=compute_training_stats.outputs.output)
2.3.4 ExampleValidator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Identifies Anomalies in Training Data Used with serving data to detect drift / skew Uses StatisticsGen and SchemaGen Outputs Produces Validation Results Uses TFDV Library for Input Validation from tfx import components infer_schema = components.SchemaGen( stats=compute_training_stats.outputs.output )
2.3.5 Transform Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uses Data from ExampleGen & SchemaGen Transformations Become Part of TF Graph (!!) Helps Avoid Training/Serving Skew Uses TFT Library for Transformations Transformations Require Full Pass Thru Dataset Global Reduction Across All Batches Create Word Embeddings, Normalize, PCA def preprocessing_fn(inputs): # inputs: map from feature keys # to raw not-yet-transformed features # outputs: map from string feature key # to transformed feature operations
2.3.6 Trainer Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Trains / Validates tf.Examples from Transform Uses schema.proto from SchemaGen Produces SavedModel and EvalSavedModel Uses Core TensorFlow Python API Works with TensorFlow 1.x Estimator API TensorFlow 2.0 Keras Support Coming Soon from tfx import components trainer = components.Trainer( module_file=taxi_pipeline_utils, train_files=transform_training.outputs.output, eval_files=transform_eval.outputs.output, schema=infer_schema.outputs.output, tf_transform_dir=transform_training.outputs.output, train_steps=10000, eval_steps=5000)
2.3.7 Evaluator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uses EvalSavedModel from Trainer Writes Analysis Results to ML Metadata Store Uses TFMA Library for Analysis TFMA Uses Apache Beam to Scale Analysis from tfx import components import tensorflow_model_analysis as tfma taxi_eval_spec = [ tfma.SingleSliceSpec(), tfma.SingleSliceSpec(columns=['trip_start_hour']) ] model_analyzer = components.Evaluator( examples=examples_gen.outputs.eval_examples, eval_spec=taxi_eval_spec, model_exports=trainer.outputs.output)
2.3.8 ModelValidator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Validate Models from Trainer Uses Data from SchemaGen & StatisticsGen Compares New Models to Baseline Baseline == current model in production New Model is Good if Meets/Exceeds Metrics If Good, Notify Pusher to Deploy New Model Simulate “Next Day Evaluation” On New Data import tensorflow_model_analysis as tfma taxi_mv_spec = [tfma.SingleSliceSpec()] model_validator = components.ModelValidator( examples=examples_gen.outputs.output, model=trainer.outputs.output)
2.3.9 Model Pusher (Deployer) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Push Good Model to Deployment Target Uses Trained SavedModel Writes Version Data to Metadata Store Write to FileSystem or TensorFlow Hub from tfx import components pusher = components.Pusher( model_export=trainer.outputs.output, model_blessing=model_validator.outputs.blessing, serving_model_dir=serving_model_dir)
2.3.10 Slack Component (!!) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Runs After ModelValidator Adds Human-in-the-Loop Step to Pipeline TFX Sends Message to Slack with Model URI Asks Human to Review the New Model Respond ‘LGTM’, ‘approve’, ‘decline’, ‘reject’ Requires Slack API Setup / Integration export SLACK_BOT_TOKEN={your_token} _channel_id = 'my-channel-id' _slack_token = os.environ['SLACK_BOT_TOKEN’] slack_validator = SlackComponent( model_export=trainer.outputs.output, model_blessing=model_validator.outputs.blessing, slack_token=_slack_token, channel_id=_channel_id, timeout_sec=3600, ) https://github.com/tensorflow/tfx/tree/master /tfx/examples/custom_components/slack/slack_component
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
3.0 ML Pipelines with Airflow and KubeFlow Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy 3.1 Airflow KubeFlow3.2
3.1 Airflow 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Most Widely-Used Workflow Orchestrator Define Execution Graphs in Python Decent UI Good Community Support
Hands On 05_Airflow_ML_Pipelines (Chicago Taxi Dataset)
Hands On 06_Airflow_Feature_Analysis
Hands On 07_Airflow_Model_Analysis
3.2 KubeFlow 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Pipelines Based on Argo CI/CD Project from Intuit TFJob & PyTorch Job Supports Distributed Training TensorFlow & PyTorch Jobs KubeFlow Fairing Project (!!) Run a notebook as a production job Deploy training code with dependencies
Hands On 08_Simple_KubeFlow_ML_Pipeline
Hands On 09_Advanced_KubeFlow_ML_Pipeline (Chicago Taxi Dataset)
Hands On 10_Distributed_TensorFlow_Job
Hands On 10a_Distributed_PyTorch_Job
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
4.0 Hyper-Parameter Tuning 6 Experiment Single Optimization Run Single Objective Function Across Runs Contains Many Trials Trial List of Param Values Suggestion Optimization Algorithm Job Evaluates a Trial Calculates Objective
Hands On 11_Hyper_Parameter_Tuning
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
5.0 Deploy Notebook as Job Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training 6 5.1 Wrap Model in a Docker Image Deploy Job to Kubernetes5.2
5.1 Create Docker Image Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
5.2 Deploy Notebook as Job Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
Hands On 12_Deploy_Notebook_Xgboost
Hands On 12a_Deploy_Notebook_TensorFlow
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with TFX, Airflow, and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with TFX and KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
7.0 MLflow 7.1 Experiment Tracking Hyper-Parameter Tuning Kubernetes-based Jobs 7.2 7.3
Hands On 14_MLflow_Scikit_Learn
Hands On 14a_MLflow_Keras
Hands On 14b_MLflow_TensorFlow
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
7.0 Papermill
Hands On 15_Papermill_Notebook_Job
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
9.0 TensorFlow Privacy (Differential Privacy)
9.1 Differential Privacy + Stochastic Gradient Descent
9.1 Differential Privacy + Explainability + Drift Detection
Hands On 16_TF_Privacy
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
10.0 Model Serving & A/B Tests
Hands On 17_Simple_Serving_REST
Hands On 17a_AB_Test_REST
Hands On 18_Metrics_and_Monitoring
10.0 Model Optimization
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
Hands On 19_Optimize_Model
Agenda – Part 2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
After Pushing Your Model to Production, Your Model is… 1 Already Out of Date – Need to Re-Train Biased – Need to Validate Before Pushing Broken – Need to A/B Test in Production Hacked – Need to Train With Data Privacy Slow – Need to Quantize and Speed Up Predictions 2 3 4 5
Community Edition https://community.pipeline.ai
Thank you! https://pipeline.ai @cfregly @PipelineAI

More Related Content

PDF
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
byAnimesh Singh
 
PPTX
Onnx and onnx runtime
byVishwas N
 
PDF
Building an ML Platform with Ray and MLflow
byDatabricks
 
PDF
MLFlow: Platform for Complete Machine Learning Lifecycle
byDatabricks
 
PDF
MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
byDatabricks
 
PPTX
Pythonsevilla2019 - Introduction to MLFlow
byFernando Ortega Gallego
 
PDF
Kubeflow
byKarane Vieira
 
PDF
MLflow Model Serving
byDatabricks
 
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
byAnimesh Singh
 
Onnx and onnx runtime
byVishwas N
 
Building an ML Platform with Ray and MLflow
byDatabricks
 
MLFlow: Platform for Complete Machine Learning Lifecycle
byDatabricks
 
MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
byDatabricks
 
Pythonsevilla2019 - Introduction to MLFlow
byFernando Ortega Gallego
 
Kubeflow
byKarane Vieira
 
MLflow Model Serving
byDatabricks
 

What's hot

PDF
Seamless MLOps with Seldon and MLflow
byDatabricks
 
PDF
MLOps with Kubeflow
bySaurabh Kaushik
 
PDF
Build an LLM-powered application using LangChain.pdf
byStephenAmell4
 
PDF
Introduction to MLflow
byDatabricks
 
PDF
What is MLOps
byHenrik Skogström
 
PDF
Transformers, LLMs, and the Possibility of AGI
bySynaptonIncorporated
 
PPTX
MLOps in action
byPieter de Bruin
 
PDF
MLOps Using MLflow
byDatabricks
 
PDF
⼤語⾔模型 LLM 應⽤開發入⾨
byWen-Tien Chang
 
PDF
Machine Learning using Kubeflow and Kubernetes
byArun Gupta
 
PDF
Build an LLM-powered application using LangChain.pdf
byAnastasiaSteele10
 
PDF
LanGCHAIN Framework
byKeymate.AI
 
PDF
Productionzing ML Model Using MLflow Model Serving
byDatabricks
 
PDF
What Is Helm
byAMELIAOLIVIA2
 
PDF
Managing the Complete Machine Learning Lifecycle with MLflow
byDatabricks
 
PDF
Basics of Generative AI: Models, Tokenization, Embeddings, Text Similarity, V...
byRobert McDermott
 
PDF
Git Branching for Agile Teams
byAtlassian
 
PPTX
DevOps
byGehad Elsayed
 
PDF
DevOps - A Gentle Introduction
byCodeOps Technologies LLP
 
PDF
Seldon: Deploying Models at Scale
bySeldon
 
Seamless MLOps with Seldon and MLflow
byDatabricks
 
MLOps with Kubeflow
bySaurabh Kaushik
 
Build an LLM-powered application using LangChain.pdf
byStephenAmell4
 
Introduction to MLflow
byDatabricks
 
What is MLOps
byHenrik Skogström
 
Transformers, LLMs, and the Possibility of AGI
bySynaptonIncorporated
 
MLOps in action
byPieter de Bruin
 
MLOps Using MLflow
byDatabricks
 
⼤語⾔模型 LLM 應⽤開發入⾨
byWen-Tien Chang
 
Machine Learning using Kubeflow and Kubernetes
byArun Gupta
 
Build an LLM-powered application using LangChain.pdf
byAnastasiaSteele10
 
LanGCHAIN Framework
byKeymate.AI
 
Productionzing ML Model Using MLflow Model Serving
byDatabricks
 
What Is Helm
byAMELIAOLIVIA2
 
Managing the Complete Machine Learning Lifecycle with MLflow
byDatabricks
 
Basics of Generative AI: Models, Tokenization, Embeddings, Text Similarity, V...
byRobert McDermott
 
Git Branching for Agile Teams
byAtlassian
 
DevOps
byGehad Elsayed
 
DevOps - A Gentle Introduction
byCodeOps Technologies LLP
 
Seldon: Deploying Models at Scale
bySeldon
 

Similar to KubeFlow + GPU + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark ML + Jupyter

PDF
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
byJan Kirenz
 
PDF
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
byChris Fregly
 
PDF
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
byAnimesh Singh
 
PDF
Tensorflow 2.0 and Coral Edge TPU
byAndrés Leonardo Martinez Ortiz
 
PDF
TensorFlow Extended: An End-to-End Machine Learning Platform for TensorFlow
byDatabricks
 
PPTX
Tensorflow Ecosystem
byVivek Raja P S
 
PDF
running Tensorflow in Production
byMatthias Feys
 
PPTX
Certification Study Group -Professional ML Engineer Session 2 (GCP-TensorFlow...
bygdgsurrey
 
PDF
Intro - End to end ML with Kubeflow @ SignalConf 2018
byHolden Karau
 
PDF
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
byBill Liu
 
PPTX
ML Ops Tools ML flow and Hugging Face(2).pptx
byMohamedHomoda3
 
PDF
Flink Forward San Francisco 2019: TensorFlow Extended: An end-to-end machine ...
byFlink Forward
 
PDF
Machine learning operations model book mlops
byRuyPerez1
 
PDF
Hopsworks at Google AI Huddle, Sunnyvale
byJim Dowling
 
PDF
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
PDF
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
PDF
TensorFlow Extension (TFX) and Apache Beam
bymarkgrover
 
PDF
Moving Your Machine Learning Models to Production with TensorFlow Extended
byJonathan Mugan
 
PDF
ML Platform Q1 Meetup: End to-end Feature Analysis, Validation and Transforma...
byFei Chen
 
PDF
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
byJan Kirenz
 
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
byJan Kirenz
 
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
byChris Fregly
 
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
byAnimesh Singh
 
Tensorflow 2.0 and Coral Edge TPU
byAndrés Leonardo Martinez Ortiz
 
TensorFlow Extended: An End-to-End Machine Learning Platform for TensorFlow
byDatabricks
 
Tensorflow Ecosystem
byVivek Raja P S
 
running Tensorflow in Production
byMatthias Feys
 
Certification Study Group -Professional ML Engineer Session 2 (GCP-TensorFlow...
bygdgsurrey
 
Intro - End to end ML with Kubeflow @ SignalConf 2018
byHolden Karau
 
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
byBill Liu
 
ML Ops Tools ML flow and Hugging Face(2).pptx
byMohamedHomoda3
 
Flink Forward San Francisco 2019: TensorFlow Extended: An end-to-end machine ...
byFlink Forward
 
Machine learning operations model book mlops
byRuyPerez1
 
Hopsworks at Google AI Huddle, Sunnyvale
byJim Dowling
 
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
TensorFlow Extension (TFX) and Apache Beam
bymarkgrover
 
Moving Your Machine Learning Models to Production with TensorFlow Extended
byJonathan Mugan
 
ML Platform Q1 Meetup: End to-end Feature Analysis, Validation and Transforma...
byFei Chen
 
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
byJan Kirenz
 

More from Chris Fregly

PDF
AWS reInvent 2022 reCap AI/ML and Data
byChris Fregly
 
PDF
Ray AI Runtime (AIR) on AWS - Data Science On AWS Meetup
byChris Fregly
 
PDF
High Performance Distributed TensorFlow in Production with GPUs - NIPS 2017 -...
byChris Fregly
 
PDF
AWS Re:Invent 2019 Re:Cap
byChris Fregly
 
PDF
Waking the Data Scientist at 2am: Detect Model Degradation on Production Mod...
byChris Fregly
 
PDF
Quantum Computing with Amazon Braket
byChris Fregly
 
PDF
High Performance TensorFlow in Production - Big Data Spain - Madrid - Nov 15 ...
byChris Fregly
 
PDF
PipelineAI + TensorFlow AI + Spark ML + Kuberenetes + Istio + AWS SageMaker +...
byChris Fregly
 
PDF
Hyper-Parameter Tuning Across the Entire AI Pipeline GPU Tech Conference San ...
byChris Fregly
 
PDF
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
byChris Fregly
 
PDF
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
byChris Fregly
 
PDF
Swift for TensorFlow - Tanmay Bakshi - Advanced Spark and TensorFlow Meetup -...
byChris Fregly
 
PDF
Smokey and the Multi-Armed Bandit Featuring BERT Reynolds Updated
byChris Fregly
 
PDF
Spark SQL Catalyst Optimizer, Custom Expressions, UDFs - Advanced Spark and T...
byChris Fregly
 
PDF
PipelineAI Real-Time Machine Learning - Global Artificial Intelligence Confer...
byChris Fregly
 
PDF
PipelineAI Continuous Machine Learning and AI - Rework Deep Learning Summit -...
byChris Fregly
 
PDF
Advanced Spark and TensorFlow Meetup - Dec 12 2017 - Dong Meng, MapR + Kubern...
byChris Fregly
 
PDF
Pandas on AWS - Let me count the ways.pdf
byChris Fregly
 
PDF
Amazon reInvent 2020 Recap: AI and Machine Learning
byChris Fregly
 
PDF
15 Tips to Scale a Large AI/ML Workshop - Both Online and In-Person
byChris Fregly
 
AWS reInvent 2022 reCap AI/ML and Data
byChris Fregly
 
Ray AI Runtime (AIR) on AWS - Data Science On AWS Meetup
byChris Fregly
 
High Performance Distributed TensorFlow in Production with GPUs - NIPS 2017 -...
byChris Fregly
 
AWS Re:Invent 2019 Re:Cap
byChris Fregly
 
Waking the Data Scientist at 2am: Detect Model Degradation on Production Mod...
byChris Fregly
 
Quantum Computing with Amazon Braket
byChris Fregly
 
High Performance TensorFlow in Production - Big Data Spain - Madrid - Nov 15 ...
byChris Fregly
 
PipelineAI + TensorFlow AI + Spark ML + Kuberenetes + Istio + AWS SageMaker +...
byChris Fregly
 
Hyper-Parameter Tuning Across the Entire AI Pipeline GPU Tech Conference San ...
byChris Fregly
 
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
byChris Fregly
 
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
byChris Fregly
 
Swift for TensorFlow - Tanmay Bakshi - Advanced Spark and TensorFlow Meetup -...
byChris Fregly
 
Smokey and the Multi-Armed Bandit Featuring BERT Reynolds Updated
byChris Fregly
 
Spark SQL Catalyst Optimizer, Custom Expressions, UDFs - Advanced Spark and T...
byChris Fregly
 
PipelineAI Real-Time Machine Learning - Global Artificial Intelligence Confer...
byChris Fregly
 
PipelineAI Continuous Machine Learning and AI - Rework Deep Learning Summit -...
byChris Fregly
 
Advanced Spark and TensorFlow Meetup - Dec 12 2017 - Dong Meng, MapR + Kubern...
byChris Fregly
 
Pandas on AWS - Let me count the ways.pdf
byChris Fregly
 
Amazon reInvent 2020 Recap: AI and Machine Learning
byChris Fregly
 
15 Tips to Scale a Large AI/ML Workshop - Both Online and In-Person
byChris Fregly
 

Recently uploaded

PDF
DevOps Monitoring Tools: The 2025 Guide to Performance & Observability
byalexendrascott01
 
PDF
Smarter Testing Safer Systems Balancing AI and Oversight in Regulated Environ...
byApplitools
 
PDF
inSis suite - Laboratory Information Management System
byKondapi V Siva Rama Brahmam
 
PDF
Bring AI and build AI agents into your Jakarta EE Apps with LangChain4J-CDI
byBuhake Sindi
 
PPTX
Presentation on AWS Cloud RDS Deep dive
byBimal Jain
 
PDF
BCA 1st Semester Fundamentals Solved Question Paper 44121
byKuvempu University
 
PPTX
The Sync Strikes Back: Tales from the MOPs Trenches
bybbedford2
 
PDF
Custom MGA Software Development: Better Apporach
byOpenkoda
 
PDF
SCORM Cloud: The 5 categories of content distribution
byRustici Software
 
PPTX
Boost Productivity the Smart Way with AI Automation Solutions
byEllocent Labs
 
PDF
DSD-INT 2025 Hydrodynamic and Morphodynamic Modeling with Delft3D FM for an I...
byDeltares
 
PDF
Presentation Empowerment Technology in ICT
byoryoseiii
 
PDF
DSD-INT 2025 Next-Generation Flood Inundation Mapping for Taiwan - Challenges...
byDeltares
 
PDF
DSD-INT 2025 Validation of SFINCS on Historical River Floods at the Global Sc...
byDeltares
 
PDF
DSD-INT 2025 Century-Scale Impacts of Sediment-Based Coastal Restoration unde...
byDeltares
 
PDF
DSD-INT 2025 Quantifying Flood Mitigation Strategies Under Sea Level Rise - H...
byDeltares
 
PDF
DSD-INT 2025 Coupling SFINCS to a flood risk model to evaluate the effects of...
byDeltares
 
PDF
DSD-INT 2025 Timor-Leste Flood exposure and Climate Change assessment - Ogunwumi
byDeltares
 
PDF
DSD-INT 2025 Building-Aware Flood and Lifeline Scour Modeling with Delft3D FM...
byDeltares
 
PDF
DSD-INT 2025 The Sinterklaas Storm in the Southwest of the Netherlands - Wope...
byDeltares
 
DevOps Monitoring Tools: The 2025 Guide to Performance & Observability
byalexendrascott01
 
Smarter Testing Safer Systems Balancing AI and Oversight in Regulated Environ...
byApplitools
 
inSis suite - Laboratory Information Management System
byKondapi V Siva Rama Brahmam
 
Bring AI and build AI agents into your Jakarta EE Apps with LangChain4J-CDI
byBuhake Sindi
 
Presentation on AWS Cloud RDS Deep dive
byBimal Jain
 
BCA 1st Semester Fundamentals Solved Question Paper 44121
byKuvempu University
 
The Sync Strikes Back: Tales from the MOPs Trenches
bybbedford2
 
Custom MGA Software Development: Better Apporach
byOpenkoda
 
SCORM Cloud: The 5 categories of content distribution
byRustici Software
 
Boost Productivity the Smart Way with AI Automation Solutions
byEllocent Labs
 
DSD-INT 2025 Hydrodynamic and Morphodynamic Modeling with Delft3D FM for an I...
byDeltares
 
Presentation Empowerment Technology in ICT
byoryoseiii
 
DSD-INT 2025 Next-Generation Flood Inundation Mapping for Taiwan - Challenges...
byDeltares
 
DSD-INT 2025 Validation of SFINCS on Historical River Floods at the Global Sc...
byDeltares
 
DSD-INT 2025 Century-Scale Impacts of Sediment-Based Coastal Restoration unde...
byDeltares
 
DSD-INT 2025 Quantifying Flood Mitigation Strategies Under Sea Level Rise - H...
byDeltares
 
DSD-INT 2025 Coupling SFINCS to a flood risk model to evaluate the effects of...
byDeltares
 
DSD-INT 2025 Timor-Leste Flood exposure and Climate Change assessment - Ogunwumi
byDeltares
 
DSD-INT 2025 Building-Aware Flood and Lifeline Scour Modeling with Delft3D FM...
byDeltares
 
DSD-INT 2025 The Sinterklaas Storm in the Southwest of the Netherlands - Wope...
byDeltares
 
In this document
Powered by AI

Overview of end-to-end ML pipelines using TFX, Kubeflow, and Airflow by Chris Fregly, founder of PipelineAI.

Interactive polling on audience's familiarity with tools like command line, Python, Docker, and Jupyter Notebooks.

Discussion of funding based on GitHub stars and promotion of the community edition of PipelineAI.

Identifies common issues after pushing models to production, including the need for retraining and validation.

Workshop agenda focusing on Kubernetes setup, TFX, ML Pipelines, hyper-parameter tuning, and deploying Notebooks.

Important notes regarding technical setup, capabilities of Kubernetes, datasets used, and troubleshooting advice.

Details on setting up the environment with TFX components and Kubernetes for ML Pipelines.

In-depth look into TFX, including components such as ExampleGen, StatisticsGen, and ModelValidator. Hands-on activities related to TFX Libraries including Data Validation and Transform processes. Technical details and hands-on activities on model analysis, validation, and deployment of ML models.

Introduction to using Airflow as a workflow orchestrator for ML pipelines, with hands-on examples.

Hands-on workshops demonstrating KubeFlow's capabilities for machine learning pipelines.

Overview of hyper-parameter tuning strategies in machine learning pipelines with practical hands-on.

Instructions on creating Docker images and deploying Jupyter notebooks as jobs in Kubernetes.

Discussion and hands-on interaction with MLflow for experiment tracking and ML jobs.

Explores differential privacy techniques and their implications in machine learning.

Final model serving, A/B testing, optimization, and hands-on activities related to monitoring.

Highlights the common issues models face post-deployment and encourages community engagement.

Closing remarks and appreciation for participation, with links to PipelineAI for further engagement.

KubeFlow + GPU + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark ML + Jupyter

  • 1.
    End-to-End ML Pipelines TFX+ Kubeflow + Airflow + MLflow Chris Fregly Founder @ .
  • 2.
    Founder @ PipelineAI ContinuousMachine Learning in Production Former Databricks, Netflix Apache Spark Contributor O’Reilly Author High Performance TensorFlow in Production Meetup Organizer Advanced Kubeflow Meetup Who Am I? (@cfregly)
  • 3.
    Advanced Kubeflow Meetup(Global+Bay Area, Monthly Events) https://meetup.com/Advanced-Kubeflow-Meetup
  • 4.
  • 5.
    1 OK withCommand Line? 2 OK with Python? 3 OK with Linear Algebra? Who are you? 4 OK with Docker? 6 5 OK with Jupyter Notebook?
  • 6.
  • 7.
    4,000 Stars =$6,000,000 Seed $1,500 per GitHub Star?! (Please star the repo ASAP!!) Recent Comment from Popular VC Investor in Silicon Valley
  • 8.
  • 9.
    After Pushing YourModel to Production, Your Model is… 1 Already Out of Date – Need to Re-Train Biased – Need to Validate Before Pushing Broken – Need to A/B Test in Production Hacked – Need to Train With Data Privacy Slow – Need to Quantize and Speed Up Predictions 2 3 4 5
  • 10.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 11.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 12.
    Note #1 of11 IGNORE WARNINGS & ERRORS (Everything will be OK!)
  • 13.
    Note #2 of11 THERE IS A LOT OF MATERIAL HERE Many opportunities to explore on your own. (Don’t upload sensitive data.)
  • 14.
    Note #3 of11 YOU HAVE YOUR OWN INSTANCE 16 CPU, 104 GB RAM, 200GB SSD (Each with a full Kubernetes Cluster.)
  • 15.
    Note #4 of11 DATASETS Chicago Taxi Dataset (and various others.)
  • 16.
    Note #5 of11 SOME NOTEBOOKS TAKE MINUTES Please be patient. (We are using large datasets)
  • 17.
    Note #6 of11 QUESTIONS? Post questions to Zoom chat or Q&A. (Antje and I will answer soon) Antje >
  • 18.
    Note #7 of11 KUBEFLOW IS NOT A SILVER BULLET There are still gaps in the pipeline. (But gaps are getting smaller)
  • 19.
    Note #8 of11 THIS IS NOT CLOUD DEPENDENT* *Except for 2 small exceptions… (Patches are underway.)
  • 20.
    Note #9 of11 PRIMARILY TENSORFLOW 1.x TF 2.x is not fully supported by TFX (Until Mid-2020.)
  • 21.
    Note #10 of11 SHUTDOWN EACH NOTEBOOK AFTER We are using complex browser voo-doo. (Javascript is a mystery.)
  • 22.
    Note #11 of11 Retrieve 1 Single IP Address Here… <INSERT HERE> (Do not click refresh.)
  • 23.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 24.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 25.
  • 26.
    1.1 Kubernetes TensorFlow Extended(TFX) Airflow ML Pipelines 1.0 Environment Overview KubeFlow ML Pipelines 6 Hyper-Parameter Tuning (Katib) Prediction Traffic Router (Istio) 1.2 1.3 1.4 1.6 1.7 MLflow Pipelines1.5
  • 27.
    1.1 Kubernetes Kubernetes NFS Ceph Cassandra MySQL Spark Airflow Tensorflow Caffe TF-Serving Flask+Scikit Operating system(Linux, Windows) CPU Memory DiskSSD GPU FPGA ASIC NIC Jupyter GCP AWS Azure On-prem Namespace Quota Logging Monitoring RBAC
  • 28.
  • 29.
    System 6 System 5System4 Training At Scale System 3 System 1 Data Ingestion Data Analysis Data Transform Data Validation System 2 Build Model Model Validation Serving Logging Monitoring Roll-out Data Splitting Ad-Hoc Training Why TFX and Why KubeFlow? Improve Training/Serving Consistency Unify Disparate Systems Manage Pipeline Complexity Improve Portability Wrangle Large Datasets Improve Model Quality Manage Versions Composability Distributed Training Configure
  • 30.
    1.2 TensorFlow Extended(TFX) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Jenkins?
  • 31.
    1.3 Airflow MLPipelines
  • 32.
    1.4 KubeFlow MLPipelines
  • 33.
  • 34.
  • 35.
    1.7 Prediction TrafficRouting (Istio)
  • 36.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 37.
    2.1 TFX Internals 2.0TFX Components 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training 2.2 TFX Libraries 2.2 TFX Components
  • 38.
    2.1 TFX Internals Driver/Publisher Movesdata to/from Metadata Store Executor Runs the Actual Processing Code Metadata Store Artifact, execution, and lineage Info Track inputs & outputs of all components Stores training run including inputs & outputs Analysis, validation, and versioning results Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  • 39.
    2.2 TFX Libraries 2.2.1 TFXComponents Use These: TensorFlow Data Validation (TFDV) TensorFlow Transform (TFT) TensorFlow Model Analysis (TFMA) TensorFlow Metadata (TFMD) + ML Metadata (MLMD) 2.2.2 2.2.3 2.2.4 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  • 40.
    2.2.1 TFX Libraries- TFDV TensorFlow Data Validation (TFDV) Find Missing, Redundant & Important Features Identify Features with Unusually-Large Scale `infer_schema()` Generates Schema Describe Feature Ranges Detect Data Drift Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uniformly Distributed Data è ç Non-Uniformly Distributed Data
  • 41.
  • 42.
    2.2.2 TFX Libraries- TFT TensorFlow Transform (TFT) Preprocess `tf.Example` data with TensorFlow Useful for data that requires a full pass Normalize all inputs by mean and std dev Create vocabulary of strings è integers over all data Bucketize features based on entire data distribution Outputs a TensorFlow graph Re-used across both training and serving Uses Apache Beam (local mode) for Parallel Analysis Can also use distributed mode `preprocessing_fn(inputs)`: Primary Fn to Implement Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training import tensorflow as tf import tensorflow_transform as tft def preprocessing_fn(inputs): x = inputs['x'] y = inputs['y'] s = inputs['s'] x_centered = x - tft.mean(x) y_normalized = tft.scale_to_0_1(y) s_integerized = tft.compute_and_apply_vocabulary(s) x_centered_times_y_normalized = x_centered * y_normalized return { 'x_centered': x_centered, 'y_normalized': y_normalized, 'x_centered_times_y_normalized':x_centered_times_y_normalized, 's_integerized': s_integerized }
  • 43.
  • 44.
  • 45.
    2.2.3 TFX Libraries- TFMA TensorFlow Model Analysis (TFMA) Analyze Model on Different Slices of Dataset Track Metrics Over Time (“Next Day Eval”) `EvalSavedModel` Contains Slicing Info TFMA Pipeline: Read, Extract, Evaluate, Write ie. Ensure Model Works Fairly Across All Users Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  • 46.
  • 47.
    2.2.4 TFX Libraries– Metadata TensorFlow Metadata (TFMD) ML Metadata (MLMD) Record and Retrieve Experiment Metadata Artifact, Execution, and Lineage Info Track Inputs / Outputs of All TFX Components Stores Training Run Info Analysis and Validation Results Model Versioning Info Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  • 48.
  • 49.
    2.3.1 ExampleGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Load Training Data Into TFX Pipeline Supports External Data Sources Supports CSV and TFRecord Formats Converts Data to tf.Example Note: TFX Pipelines require tf.Example (?!) Difficult to use non-TF models like XGBoost from tfx.utils.dsl_utils import csv_input from tfx.components.example_gen.csv_example_gen.component import CsvExampleGen examples = csv_input(os.path.join(base_dir, 'data/simple')) example_gen = CsvExampleGen(input_base=examples)
  • 50.
    2.3.2 StatisticsGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Generates Statistics on Training Data Global `mean` and `stddev` per input feature Consumes tf.Example instances from tfx import components compute_eval_stats = components.StatisticsGen( input_data=examples_gen.outputs.eval_examples, name='compute-eval-stats' )
  • 51.
    2.3.3 SchemaGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Schema Needed by Some TFX Components Data Types, Value Ranges, Optional, Required Consumes Data from StatisticsGen Schema used by TFDV, TFT, TFMA Libraries Uses TFDV Library to infer schema Best effort and basic Human should verify feature { name: "age" value_count { min: 1 max: 1 } type: FLOAT presence { min_fraction: 1 min_count: 1 } } from tfx import components infer_schema = components.SchemaGen( stats=compute_training_stats.outputs.output)
  • 52.
    2.3.4 ExampleValidator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Identifies Anomalies in Training Data Used with serving data to detect drift / skew Uses StatisticsGen and SchemaGen Outputs Produces Validation Results Uses TFDV Library for Input Validation from tfx import components infer_schema = components.SchemaGen( stats=compute_training_stats.outputs.output )
  • 53.
    2.3.5 Transform Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Uses Data from ExampleGen & SchemaGen Transformations Become Part of TF Graph (!!) Helps Avoid Training/Serving Skew Uses TFT Library for Transformations Transformations Require Full Pass Thru Dataset Global Reduction Across All Batches Create Word Embeddings, Normalize, PCA def preprocessing_fn(inputs): # inputs: map from feature keys # to raw not-yet-transformed features # outputs: map from string feature key # to transformed feature operations
  • 54.
    2.3.6 Trainer Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Trains / Validates tf.Examples from Transform Uses schema.proto from SchemaGen Produces SavedModel and EvalSavedModel Uses Core TensorFlow Python API Works with TensorFlow 1.x Estimator API TensorFlow 2.0 Keras Support Coming Soon from tfx import components trainer = components.Trainer( module_file=taxi_pipeline_utils, train_files=transform_training.outputs.output, eval_files=transform_eval.outputs.output, schema=infer_schema.outputs.output, tf_transform_dir=transform_training.outputs.output, train_steps=10000, eval_steps=5000)
  • 55.
    2.3.7 Evaluator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Uses EvalSavedModel from Trainer Writes Analysis Results to ML Metadata Store Uses TFMA Library for Analysis TFMA Uses Apache Beam to Scale Analysis from tfx import components import tensorflow_model_analysis as tfma taxi_eval_spec = [ tfma.SingleSliceSpec(), tfma.SingleSliceSpec(columns=['trip_start_hour']) ] model_analyzer = components.Evaluator( examples=examples_gen.outputs.eval_examples, eval_spec=taxi_eval_spec, model_exports=trainer.outputs.output)
  • 56.
    2.3.8 ModelValidator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Validate Models from Trainer Uses Data from SchemaGen & StatisticsGen Compares New Models to Baseline Baseline == current model in production New Model is Good if Meets/Exceeds Metrics If Good, Notify Pusher to Deploy New Model Simulate “Next Day Evaluation” On New Data import tensorflow_model_analysis as tfma taxi_mv_spec = [tfma.SingleSliceSpec()] model_validator = components.ModelValidator( examples=examples_gen.outputs.output, model=trainer.outputs.output)
  • 57.
    2.3.9 Model Pusher(Deployer) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Push Good Model to Deployment Target Uses Trained SavedModel Writes Version Data to Metadata Store Write to FileSystem or TensorFlow Hub from tfx import components pusher = components.Pusher( model_export=trainer.outputs.output, model_blessing=model_validator.outputs.blessing, serving_model_dir=serving_model_dir)
  • 58.
    2.3.10 Slack Component(!!) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Runs After ModelValidator Adds Human-in-the-Loop Step to Pipeline TFX Sends Message to Slack with Model URI Asks Human to Review the New Model Respond ‘LGTM’, ‘approve’, ‘decline’, ‘reject’ Requires Slack API Setup / Integration export SLACK_BOT_TOKEN={your_token} _channel_id = 'my-channel-id' _slack_token = os.environ['SLACK_BOT_TOKEN’] slack_validator = SlackComponent( model_export=trainer.outputs.output, model_blessing=model_validator.outputs.blessing, slack_token=_slack_token, channel_id=_channel_id, timeout_sec=3600, ) https://github.com/tensorflow/tfx/tree/master /tfx/examples/custom_components/slack/slack_component
  • 59.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 60.
    3.0 ML Pipelineswith Airflow and KubeFlow Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy 3.1 Airflow KubeFlow3.2
  • 61.
    3.1 Airflow 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Most Widely-Used Workflow Orchestrator Define Execution Graphs in Python Decent UI Good Community Support
  • 62.
  • 63.
  • 64.
  • 65.
    3.2 KubeFlow 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model DeployReproduce Training Pipelines Based on Argo CI/CD Project from Intuit TFJob & PyTorch Job Supports Distributed Training TensorFlow & PyTorch Jobs KubeFlow Fairing Project (!!) Run a notebook as a production job Deploy training code with dependencies
  • 66.
  • 67.
  • 68.
  • 69.
  • 70.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 71.
    4.0 Hyper-Parameter Tuning 6 Experiment SingleOptimization Run Single Objective Function Across Runs Contains Many Trials Trial List of Param Values Suggestion Optimization Algorithm Job Evaluates a Trial Calculates Objective
  • 72.
  • 73.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 74.
    5.0 Deploy Notebookas Job Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training 6 5.1 Wrap Model in a Docker Image Deploy Job to Kubernetes5.2
  • 75.
    5.1 Create DockerImage Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  • 76.
    5.2 Deploy Notebookas Job Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  • 77.
  • 78.
  • 79.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with TFX, Airflow, and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with TFX and KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 80.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 81.
    7.0 MLflow 7.1 ExperimentTracking Hyper-Parameter Tuning Kubernetes-based Jobs 7.2 7.3
  • 82.
  • 83.
  • 84.
  • 85.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 86.
  • 87.
  • 88.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 89.
    9.0 TensorFlow Privacy(Differential Privacy)
  • 90.
    9.1 Differential Privacy+ Stochastic Gradient Descent
  • 91.
    9.1 Differential Privacy+ Explainability + Drift Detection
  • 92.
  • 93.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 94.
    10.0 Model Serving& A/B Tests
  • 95.
  • 96.
  • 97.
  • 98.
  • 99.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 100.
  • 101.
    Agenda – Part2 of 2 6 MLflow 7 8 9 10 TensorFlow Privacy Model Serving & A/B Tests Model Optimization Papermill
  • 102.
    1 Setup Environmentwith Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda – Part 1 of 2 Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  • 103.
    After Pushing YourModel to Production, Your Model is… 1 Already Out of Date – Need to Re-Train Biased – Need to Validate Before Pushing Broken – Need to A/B Test in Production Hacked – Need to Train With Data Privacy Slow – Need to Quantize and Speed Up Predictions 2 3 4 5
  • 104.
  • 105.