SlideShare a Scribd company logo
1 of 48
Download to read offline
The Function, the Context, and the Data
Building an Abstraction for Simpler ML Ops at Stitch Fix
Elijah ben Izzy
Data Platform Engineer - Model Lifecycle
@elijahbenizzy
linkedin.com/in/elijahbenizzy
Try out Stitch Fix → goo.gl/Q3tCQ3
2
- Stitch Fix/Data Science (DS) @ Stitch Fix
- Common Workflows/Motivation
- Representing a Model
- Unlocked Capabilities
- Future Musings
Agenda
3
The right abstraction enables separation of concerns between DS and Platforms
Take Home
DAIS 2021 4
whoami
Stitch Fix?
DAIS 2021 6
Stitch Fix is a Personal Styling Service
Shop at your personal curated store. Check out what you like.
DAIS 2021 7
Data Science is Behind Everything We Do
algorithms-tour.stitchfix.com
Algorithms Org.
- 145+ Data Scientists and Platform Engineers
- 3 main verticals + platform
Data Platform
Data Science
@ Stitch Fix
DAIS 2021 9
Common Approaches to Data Science
Typical organization:
● Horizontal teams
● Hand off between fns
● Coordination required
DATA SCIENCE /
RESEARCH TEAMS
ETL TEAMS
ENGINEERING TEAMS
DAIS 2021 10
At Stitch Fix:
● Single organization
● No handoffs
● End to end ownership
● Lots of DS!
● Built on top of data
platform tools &
abstractions
Data Scientists (DS) are Full Stack
See https://cultivating-algos.stitchfix.com/
DATA SCIENCE
ETL
ENGINEERING
The Problem
DAIS 2021 12
The Problem with Verticals
“DS are full stack” != “DS builds stack from the ground up”
Goal: scale without
-> more complex infrastructure
-> more cognitive burden on DS
DS should always be full stack... ...but can we shorten the stack?
ML platform
DAIS 2021 13
Examining Workflows
etl.py save on s3
copy to
production
Training (run at a regular cadence)
Inference
model
microservice
predictions in
batch
streaming
predictions
track metrics
share with other
teams
Analysis
DAIS 2021 14
Optimizing the Workflow
Goal: Build abstraction to give DS all these capabilities for free
Caveat: Largely uniform workflows with independent technologies
???????
model
microservice
predictions in
batch
streaming
predictions
track metrics
share with other
teams
...
The Lede
DAIS 2021 16
Build or Buy?
We built our own
- Seamless integration with current infrastructure -> leverage
- Model tracking/management data model was not standard
- We have lots of segments/varying ways to slice and dice our models
- Custom build allows for pivoting as needed
- Invest in interface design to allow for plug/play with open-source options
Called it the Model Envelope
Hats off to MLFlow, TFX, modelDB!
DAIS 2021 17
What we Built
DS only writes training script -- the rest is configuration-driven
import model_envelope as me
from sklearn import linear_model, metrics
df_X, df_y = load_data_somehow()
model = linear_model.LogisticRegression(multi_class='auto')
model.fit(df_X, df_y)
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X, api_output=df_y,
tags={'canonical_name':'foo-bar'})
my_envelope.log_metrics(validation_loss=metrics.log_loss(df_X, df_y))
DAIS 2021 18
Model Envelope (ctd.)
model
microservice
predictions in
batch
streaming
predictions
track metrics
...
share with other
teams
model
envelope
registry
Representing a Model
DAIS 2021 20
Writing a Recipe
The instructions
The cookware
The ingredients
DAIS 2021 21
Representing a Model
The function: what the model does
The context: where/how to run the model
The data: data the model needs to run
DAIS 2021 22
The Function
Artifact + Shape
DAIS 2021 23
The Function
Artifact + Shape
- Serialized model (bytes) including state
- Serialization metadata
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X,
api_output=df_y,
tags={'canonical_name':'foo-bar'})
DS passes object, platform serializes
Platform derives metadata
DAIS 2021 24
The Function
Artifact + Shape
- Function inputs
- Function outputs
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X,
api_output=df_y,
tags={'canonical_name':'foo-bar'})
DS passes sample dataframe or specifies type-annotations
Platform serializes, represents in custom format
DAIS 2021 25
The Context
Environment + Index
DAIS 2021 26
The Context
Environment + Index
- Installed packages
- Custom code
- Language + version
import my_custom_fancy_ml_module
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X,
api_output=df_y,
tags={'canonical_name':'foo-bar'},
# pip_env=['scikit-learn', pandas'], edge case if needed
custom_modules=[my_custom_fancy_ml_module])
Platform automagically derived, or DS passes pointers
DS passes in as needed
Platform automagically derived
DAIS 2021 27
The Context
Environment + Index
- Key-value tags
- Spine/index of envelope registry
import my_custom_fancy_ml_module
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X,
api_output=df_y,
tags={'canonical_name':'foo-bar'},
custom_modules=[my_custom_module])
Platform derives base tags
DS passes custom tags as desired
`
DAIS 2021 28
The Data
Training Data + Metrics
DAIS 2021 29
The Data
Training Data + Metrics
- Features
- Summary statistics
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X,
api_output=df_y,
feature_store_pointers=...)
DS (optionally) passes spec for features
Platform derives summary stats from passed data
DAIS 2021 30
The Data
Training Data + Metrics
- Scalars
- Fancy metrics
my_envelope = me.save_model(instance_name='my_model_instance_name',
instance_description='my_model_instance_description',
model=model,
query_function='predict',
api_input=df_X,
api_output=df_y,
feature_store_pointers=...)
evaluations = model(df_X)
my_envelope.log_metrics(
validation_loss=metrics.log_loss(evaluations, df_y)
roc_curve=metrics.roc_curve(evaluations, df_y))
)
DS logs metrics
using Platform metric-schema library
Unlocked Capabilities
DAIS 2021 32
Online Inference
Approach Generate, automatically deploy microservice for model predictions
1. Runs cron job to determine models for deployment
2. Generates code to run model microservice
3. Deploys models with config to AWS
4. Monitors/manages model infrastructure
1. Generates, tests out service locally
2. Sets up automatic deployment “rule”
3. Publishes model, waits
DS Platform
DAIS 2021 33
Online Inference
The Function
- Serialized artifacted loaded on service instantiation, called during endpoints
- Function shape used to create OpenAPI spec/validate inputs
DAIS 2021 34
Online Inference
The Context
- Tag spec used to automatically deploy whenever new model is published
- Note: user never has to call deploy()! Done through system-managed CD.
- Stored package versions used to build docker images
- Custom code made accessible to model for deserialization, execution
Docker Image
installed python
packages
custom code
CD
DAIS 2021 35
Online Inference
The Data
- Summary stats used to validate/monitor input (data drift)
- Feature pointer used to load feature data
Feature Store
DAIS 2021 36
Batch Inference
Approach Generate batch job in Stitch Fix workflow system (on top of airflow/flotilla)
1. Spins up spark cluster (if specified)
2. Loads input data, optionally joins with features
3. Execute model’s predict function over input
4. Saves to output table
1. Creates config for batch job (local/spark)
a. tag query to choose model
b. input/output tables
2. Executes as part of ETL
DS Platform
DAIS 2021 37
Batch Inference
The function
- Serialized artifacted loaded on batch job start
- Function shape used to validate against inputs and outputs
- MapPartitions + Pyarrow used to run models that take in DFs efficiently on spark -- abstracted away from user
DAIS 2021 38
Batch Inference
The context
- Frozen package, language versions used in installing dependencies
- Custom code made accessible to model for deserialization, execution
- Tags used to determine which model to run
Docker Image
installed python
packages
custom code
DAIS 2021 39
Batch Inference
The data
- Feature pointer used to load feature data if IDs specified
- Evaluation table pointers stored in the registry
Feature Store
DAIS 2021 40
Metrics Tracking
Approach Allow for metrics tracking with tag-based querying
1. Builds/manages dashboard
2. Adds fancy new metric types!
1. Logs metrics using python client
2. Explores in the Model Operations Dashboard
3. Saves URL for favorite viz
DS Platform
DAIS 2021 41
Metrics Tracking
DAIS 2021 42
Metrics Tracking
DAIS 2021 43
Metrics Tracking
In Summation
DAIS 2021 45
Value Added by Separating Concerns
Making deployment easy
Ensuring environment in prod == environment in training
Providing easy metrics analysis
Wrapping up complex systems
Behind-the-scenes best practices
Creating the best model
Choosing the best libraries
Determining the right metrics to log
DS concerned with... Platform concerned with...
DS focuses on creating the best model [writing the recipe]
Platform focuses on optimal infrastructure [cooking it]
Future Musings
DAIS 2021 47
Some Ideas...
More advanced use of the data
- production monitoring: utilize training data/stats to have visibility into prod/training drift
More deployment contexts
- Predictions on streaming/kafka topics
More sophisticated feature tracking/integration
- Feature stores are all the rage…
Lambda-like architecture
- Rather than requiring a deploy, can we query system for a model’s predictions?
- Requires more unified environments…
Attach external capabilities to replace home-built components of our own system...
Questions?
Find me at:
@elijahbenizzy
linkedin.com/in/elijahbenizzy
elijah.benizzy@stitchfix.com
Try out Stitch Fix → goo.gl/Q3tCQ3

More Related Content

What's hot

Data Discovery at Databricks with Amundsen
Data Discovery at Databricks with AmundsenData Discovery at Databricks with Amundsen
Data Discovery at Databricks with AmundsenDatabricks
 
Advanced SQL For Data Scientists
Advanced SQL For Data ScientistsAdvanced SQL For Data Scientists
Advanced SQL For Data ScientistsDatabricks
 
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowBuilding an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowDatabricks
 
Apache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise CompanyApache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise CompanyDatabricks
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentDatabricks
 
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)Cathrine Wilhelmsen
 
Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSA
Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSABuilding the Foundations of an Intelligent, Event-Driven Data Platform at EFSA
Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSADatabricks
 
Delight: An Improved Apache Spark UI, Free, and Cross-Platform
Delight: An Improved Apache Spark UI, Free, and Cross-PlatformDelight: An Improved Apache Spark UI, Free, and Cross-Platform
Delight: An Improved Apache Spark UI, Free, and Cross-PlatformDatabricks
 
Harnessing Spark Catalyst for Custom Data Payloads
Harnessing Spark Catalyst for Custom Data PayloadsHarnessing Spark Catalyst for Custom Data Payloads
Harnessing Spark Catalyst for Custom Data PayloadsSimeon Fitch
 
Azure Data Lake Analytics Deep Dive
Azure Data Lake Analytics Deep DiveAzure Data Lake Analytics Deep Dive
Azure Data Lake Analytics Deep DiveIlyas F ☁☁☁
 
The Critical Missing Component in the Production ML Stack
The Critical Missing Component in the Production ML StackThe Critical Missing Component in the Production ML Stack
The Critical Missing Component in the Production ML StackDatabricks
 
Machine Learning Data Lineage with MLflow and Delta Lake
Machine Learning Data Lineage with MLflow and Delta LakeMachine Learning Data Lineage with MLflow and Delta Lake
Machine Learning Data Lineage with MLflow and Delta LakeDatabricks
 
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. NielsenJ1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. NielsenMS Cloud Summit
 
Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...
Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...
Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...Databricks
 
Azure data analytics platform - A reference architecture
Azure data analytics platform - A reference architecture Azure data analytics platform - A reference architecture
Azure data analytics platform - A reference architecture Rajesh Kumar
 
Azure Data Lake and U-SQL
Azure Data Lake and U-SQLAzure Data Lake and U-SQL
Azure Data Lake and U-SQLMichael Rys
 
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
MLOps with a Feature Store: Filling the Gap in ML InfrastructureMLOps with a Feature Store: Filling the Gap in ML Infrastructure
MLOps with a Feature Store: Filling the Gap in ML InfrastructureData Science Milan
 
Building End-to-End Delta Pipelines on GCP
Building End-to-End Delta Pipelines on GCPBuilding End-to-End Delta Pipelines on GCP
Building End-to-End Delta Pipelines on GCPDatabricks
 
Microsoft Azure Databricks
Microsoft Azure DatabricksMicrosoft Azure Databricks
Microsoft Azure DatabricksSascha Dittmann
 

What's hot (20)

Data Discovery at Databricks with Amundsen
Data Discovery at Databricks with AmundsenData Discovery at Databricks with Amundsen
Data Discovery at Databricks with Amundsen
 
Advanced SQL For Data Scientists
Advanced SQL For Data ScientistsAdvanced SQL For Data Scientists
Advanced SQL For Data Scientists
 
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowBuilding an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflow
 
Apache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise CompanyApache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise Company
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload Deployment
 
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
 
Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSA
Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSABuilding the Foundations of an Intelligent, Event-Driven Data Platform at EFSA
Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSA
 
Delight: An Improved Apache Spark UI, Free, and Cross-Platform
Delight: An Improved Apache Spark UI, Free, and Cross-PlatformDelight: An Improved Apache Spark UI, Free, and Cross-Platform
Delight: An Improved Apache Spark UI, Free, and Cross-Platform
 
Harnessing Spark Catalyst for Custom Data Payloads
Harnessing Spark Catalyst for Custom Data PayloadsHarnessing Spark Catalyst for Custom Data Payloads
Harnessing Spark Catalyst for Custom Data Payloads
 
Azure Data Lake Analytics Deep Dive
Azure Data Lake Analytics Deep DiveAzure Data Lake Analytics Deep Dive
Azure Data Lake Analytics Deep Dive
 
The Critical Missing Component in the Production ML Stack
The Critical Missing Component in the Production ML StackThe Critical Missing Component in the Production ML Stack
The Critical Missing Component in the Production ML Stack
 
Machine Learning Data Lineage with MLflow and Delta Lake
Machine Learning Data Lineage with MLflow and Delta LakeMachine Learning Data Lineage with MLflow and Delta Lake
Machine Learning Data Lineage with MLflow and Delta Lake
 
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. NielsenJ1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
 
An intro to Azure Data Lake
An intro to Azure Data LakeAn intro to Azure Data Lake
An intro to Azure Data Lake
 
Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...
Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...
Designing ETL Pipelines with Structured Streaming and Delta Lake—How to Archi...
 
Azure data analytics platform - A reference architecture
Azure data analytics platform - A reference architecture Azure data analytics platform - A reference architecture
Azure data analytics platform - A reference architecture
 
Azure Data Lake and U-SQL
Azure Data Lake and U-SQLAzure Data Lake and U-SQL
Azure Data Lake and U-SQL
 
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
MLOps with a Feature Store: Filling the Gap in ML InfrastructureMLOps with a Feature Store: Filling the Gap in ML Infrastructure
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
 
Building End-to-End Delta Pipelines on GCP
Building End-to-End Delta Pipelines on GCPBuilding End-to-End Delta Pipelines on GCP
Building End-to-End Delta Pipelines on GCP
 
Microsoft Azure Databricks
Microsoft Azure DatabricksMicrosoft Azure Databricks
Microsoft Azure Databricks
 

Similar to The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

Data modeling star schema
Data modeling star schemaData modeling star schema
Data modeling star schemaSayed Ahmed
 
Be a database professional
Be a database professionalBe a database professional
Be a database professionalSayed Ahmed
 
Be a database professional
Be a database professionalBe a database professional
Be a database professionalSayed Ahmed
 
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkDatabricks
 
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...Data Con LA
 
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...Piyush Kumar
 
Developing and deploying AI solutions on the cloud using Team Data Science Pr...
Developing and deploying AI solutions on the cloud using Team Data Science Pr...Developing and deploying AI solutions on the cloud using Team Data Science Pr...
Developing and deploying AI solutions on the cloud using Team Data Science Pr...Debraj GuhaThakurta
 
Evolutionary db development
Evolutionary db development Evolutionary db development
Evolutionary db development Open Party
 
Azure machine learning service
Azure machine learning serviceAzure machine learning service
Azure machine learning serviceRuth Yakubu
 
Salesforce Analytics Cloud - Explained
Salesforce Analytics Cloud - ExplainedSalesforce Analytics Cloud - Explained
Salesforce Analytics Cloud - ExplainedCarl Brundage
 
[DSC Europe 22] Smart approach in development and deployment process for vari...
[DSC Europe 22] Smart approach in development and deployment process for vari...[DSC Europe 22] Smart approach in development and deployment process for vari...
[DSC Europe 22] Smart approach in development and deployment process for vari...DataScienceConferenc1
 
Wix's ML Platform
Wix's ML PlatformWix's ML Platform
Wix's ML PlatformRan Romano
 
ML Infrastracture @ Dropbox
ML Infrastracture @ Dropbox ML Infrastracture @ Dropbox
ML Infrastracture @ Dropbox Tsahi Glik
 
Hamburg Data Science Meetup - MLOps with a Feature Store
Hamburg Data Science Meetup - MLOps with a Feature StoreHamburg Data Science Meetup - MLOps with a Feature Store
Hamburg Data Science Meetup - MLOps with a Feature StoreMoritz Meister
 
"Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow""Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow"Databricks
 
Dowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptx
Dowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptxDowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptx
Dowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptxLex Avstreikh
 
Building a Feature Store around Dataframes and Apache Spark
Building a Feature Store around Dataframes and Apache SparkBuilding a Feature Store around Dataframes and Apache Spark
Building a Feature Store around Dataframes and Apache SparkDatabricks
 
Simplify Feature Engineering in Your Data Warehouse
Simplify Feature Engineering in Your Data WarehouseSimplify Feature Engineering in Your Data Warehouse
Simplify Feature Engineering in Your Data WarehouseFeatureByte
 
Data Mining for Developers
Data Mining for DevelopersData Mining for Developers
Data Mining for Developersllangit
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...James Anderson
 

Similar to The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix (20)

Data modeling star schema
Data modeling star schemaData modeling star schema
Data modeling star schema
 
Be a database professional
Be a database professionalBe a database professional
Be a database professional
 
Be a database professional
Be a database professionalBe a database professional
Be a database professional
 
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache Spark
 
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
 
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
 
Developing and deploying AI solutions on the cloud using Team Data Science Pr...
Developing and deploying AI solutions on the cloud using Team Data Science Pr...Developing and deploying AI solutions on the cloud using Team Data Science Pr...
Developing and deploying AI solutions on the cloud using Team Data Science Pr...
 
Evolutionary db development
Evolutionary db development Evolutionary db development
Evolutionary db development
 
Azure machine learning service
Azure machine learning serviceAzure machine learning service
Azure machine learning service
 
Salesforce Analytics Cloud - Explained
Salesforce Analytics Cloud - ExplainedSalesforce Analytics Cloud - Explained
Salesforce Analytics Cloud - Explained
 
[DSC Europe 22] Smart approach in development and deployment process for vari...
[DSC Europe 22] Smart approach in development and deployment process for vari...[DSC Europe 22] Smart approach in development and deployment process for vari...
[DSC Europe 22] Smart approach in development and deployment process for vari...
 
Wix's ML Platform
Wix's ML PlatformWix's ML Platform
Wix's ML Platform
 
ML Infrastracture @ Dropbox
ML Infrastracture @ Dropbox ML Infrastracture @ Dropbox
ML Infrastracture @ Dropbox
 
Hamburg Data Science Meetup - MLOps with a Feature Store
Hamburg Data Science Meetup - MLOps with a Feature StoreHamburg Data Science Meetup - MLOps with a Feature Store
Hamburg Data Science Meetup - MLOps with a Feature Store
 
"Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow""Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow"
 
Dowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptx
Dowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptxDowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptx
Dowling buso-feature-store-logical-clocks-spark-ai-summit-2020.pptx
 
Building a Feature Store around Dataframes and Apache Spark
Building a Feature Store around Dataframes and Apache SparkBuilding a Feature Store around Dataframes and Apache Spark
Building a Feature Store around Dataframes and Apache Spark
 
Simplify Feature Engineering in Your Data Warehouse
Simplify Feature Engineering in Your Data WarehouseSimplify Feature Engineering in Your Data Warehouse
Simplify Feature Engineering in Your Data Warehouse
 
Data Mining for Developers
Data Mining for DevelopersData Mining for Developers
Data Mining for Developers
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
 

More from Databricks

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDatabricks
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Databricks
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Databricks
 
Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Databricks
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Databricks
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of HadoopDatabricks
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDatabricks
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceDatabricks
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringDatabricks
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationDatabricks
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchDatabricks
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesDatabricks
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesDatabricks
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsDatabricks
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkDatabricks
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkDatabricks
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkDatabricks
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeDatabricks
 
Machine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionMachine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionDatabricks
 
Jeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and QualityJeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and QualityDatabricks
 

More from Databricks (20)

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2
 
Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature Aggregations
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
 
Machine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionMachine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack Detection
 
Jeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and QualityJeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and Quality
 

Recently uploaded

一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理pyhepag
 
2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Calllward7
 
Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)Jon Hansen
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyRafigAliyev2
 
Toko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra Malang
Toko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra MalangToko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra Malang
Toko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra Malangadet6151
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证ppy8zfkfm
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...ssuserf63bd7
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfRobertoOcampo24
 
社内勉強会資料  Mamba - A new era or ephemeral
社内勉強会資料   Mamba - A new era or ephemeral社内勉強会資料   Mamba - A new era or ephemeral
社内勉強会資料  Mamba - A new era or ephemeralNABLAS株式会社
 
How I opened a fake bank account and didn't go to prison
How I opened a fake bank account and didn't go to prisonHow I opened a fake bank account and didn't go to prison
How I opened a fake bank account and didn't go to prisonPayment Village
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理pyhepag
 
Pre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxPre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxStephen266013
 
Easy and simple project file on mp online
Easy and simple project file on mp onlineEasy and simple project file on mp online
Easy and simple project file on mp onlinebalibahu1313
 
Artificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfArtificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfscitechtalktv
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Valters Lauzums
 
ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp Number 24/7
ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp  Number 24/7ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp  Number 24/7
ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp Number 24/7gragkhusi
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...BabaJohn3
 

Recently uploaded (20)

一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理
 
2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call
 
Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertainty
 
Toko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra Malang
Toko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra MalangToko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra Malang
Toko Jual Viagra Asli Di Malang 081229400522 COD Obat Kuat Viagra Malang
 
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotecAbortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
社内勉強会資料  Mamba - A new era or ephemeral
社内勉強会資料   Mamba - A new era or ephemeral社内勉強会資料   Mamba - A new era or ephemeral
社内勉強会資料  Mamba - A new era or ephemeral
 
How I opened a fake bank account and didn't go to prison
How I opened a fake bank account and didn't go to prisonHow I opened a fake bank account and didn't go to prison
How I opened a fake bank account and didn't go to prison
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
 
Pre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxPre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptx
 
Easy and simple project file on mp online
Easy and simple project file on mp onlineEasy and simple project file on mp online
Easy and simple project file on mp online
 
Artificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfArtificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdf
 
Machine Learning for Accident Severity Prediction
Machine Learning for Accident Severity PredictionMachine Learning for Accident Severity Prediction
Machine Learning for Accident Severity Prediction
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
 
ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp Number 24/7
ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp  Number 24/7ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp  Number 24/7
ℂall Girls Balbir Nagar ℂall Now Chhaya ☎ 9899900591 WhatsApp Number 24/7
 
123.docx. .
123.docx.                                 .123.docx.                                 .
123.docx. .
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
 

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

  • 1. The Function, the Context, and the Data Building an Abstraction for Simpler ML Ops at Stitch Fix Elijah ben Izzy Data Platform Engineer - Model Lifecycle @elijahbenizzy linkedin.com/in/elijahbenizzy Try out Stitch Fix → goo.gl/Q3tCQ3
  • 2. 2 - Stitch Fix/Data Science (DS) @ Stitch Fix - Common Workflows/Motivation - Representing a Model - Unlocked Capabilities - Future Musings Agenda
  • 3. 3 The right abstraction enables separation of concerns between DS and Platforms Take Home
  • 6. DAIS 2021 6 Stitch Fix is a Personal Styling Service Shop at your personal curated store. Check out what you like.
  • 7. DAIS 2021 7 Data Science is Behind Everything We Do algorithms-tour.stitchfix.com Algorithms Org. - 145+ Data Scientists and Platform Engineers - 3 main verticals + platform Data Platform
  • 9. DAIS 2021 9 Common Approaches to Data Science Typical organization: ● Horizontal teams ● Hand off between fns ● Coordination required DATA SCIENCE / RESEARCH TEAMS ETL TEAMS ENGINEERING TEAMS
  • 10. DAIS 2021 10 At Stitch Fix: ● Single organization ● No handoffs ● End to end ownership ● Lots of DS! ● Built on top of data platform tools & abstractions Data Scientists (DS) are Full Stack See https://cultivating-algos.stitchfix.com/ DATA SCIENCE ETL ENGINEERING
  • 12. DAIS 2021 12 The Problem with Verticals “DS are full stack” != “DS builds stack from the ground up” Goal: scale without -> more complex infrastructure -> more cognitive burden on DS DS should always be full stack... ...but can we shorten the stack? ML platform
  • 13. DAIS 2021 13 Examining Workflows etl.py save on s3 copy to production Training (run at a regular cadence) Inference model microservice predictions in batch streaming predictions track metrics share with other teams Analysis
  • 14. DAIS 2021 14 Optimizing the Workflow Goal: Build abstraction to give DS all these capabilities for free Caveat: Largely uniform workflows with independent technologies ??????? model microservice predictions in batch streaming predictions track metrics share with other teams ...
  • 16. DAIS 2021 16 Build or Buy? We built our own - Seamless integration with current infrastructure -> leverage - Model tracking/management data model was not standard - We have lots of segments/varying ways to slice and dice our models - Custom build allows for pivoting as needed - Invest in interface design to allow for plug/play with open-source options Called it the Model Envelope Hats off to MLFlow, TFX, modelDB!
  • 17. DAIS 2021 17 What we Built DS only writes training script -- the rest is configuration-driven import model_envelope as me from sklearn import linear_model, metrics df_X, df_y = load_data_somehow() model = linear_model.LogisticRegression(multi_class='auto') model.fit(df_X, df_y) my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, tags={'canonical_name':'foo-bar'}) my_envelope.log_metrics(validation_loss=metrics.log_loss(df_X, df_y))
  • 18. DAIS 2021 18 Model Envelope (ctd.) model microservice predictions in batch streaming predictions track metrics ... share with other teams model envelope registry
  • 20. DAIS 2021 20 Writing a Recipe The instructions The cookware The ingredients
  • 21. DAIS 2021 21 Representing a Model The function: what the model does The context: where/how to run the model The data: data the model needs to run
  • 22. DAIS 2021 22 The Function Artifact + Shape
  • 23. DAIS 2021 23 The Function Artifact + Shape - Serialized model (bytes) including state - Serialization metadata my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, tags={'canonical_name':'foo-bar'}) DS passes object, platform serializes Platform derives metadata
  • 24. DAIS 2021 24 The Function Artifact + Shape - Function inputs - Function outputs my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, tags={'canonical_name':'foo-bar'}) DS passes sample dataframe or specifies type-annotations Platform serializes, represents in custom format
  • 25. DAIS 2021 25 The Context Environment + Index
  • 26. DAIS 2021 26 The Context Environment + Index - Installed packages - Custom code - Language + version import my_custom_fancy_ml_module my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, tags={'canonical_name':'foo-bar'}, # pip_env=['scikit-learn', pandas'], edge case if needed custom_modules=[my_custom_fancy_ml_module]) Platform automagically derived, or DS passes pointers DS passes in as needed Platform automagically derived
  • 27. DAIS 2021 27 The Context Environment + Index - Key-value tags - Spine/index of envelope registry import my_custom_fancy_ml_module my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, tags={'canonical_name':'foo-bar'}, custom_modules=[my_custom_module]) Platform derives base tags DS passes custom tags as desired `
  • 28. DAIS 2021 28 The Data Training Data + Metrics
  • 29. DAIS 2021 29 The Data Training Data + Metrics - Features - Summary statistics my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, feature_store_pointers=...) DS (optionally) passes spec for features Platform derives summary stats from passed data
  • 30. DAIS 2021 30 The Data Training Data + Metrics - Scalars - Fancy metrics my_envelope = me.save_model(instance_name='my_model_instance_name', instance_description='my_model_instance_description', model=model, query_function='predict', api_input=df_X, api_output=df_y, feature_store_pointers=...) evaluations = model(df_X) my_envelope.log_metrics( validation_loss=metrics.log_loss(evaluations, df_y) roc_curve=metrics.roc_curve(evaluations, df_y)) ) DS logs metrics using Platform metric-schema library
  • 32. DAIS 2021 32 Online Inference Approach Generate, automatically deploy microservice for model predictions 1. Runs cron job to determine models for deployment 2. Generates code to run model microservice 3. Deploys models with config to AWS 4. Monitors/manages model infrastructure 1. Generates, tests out service locally 2. Sets up automatic deployment “rule” 3. Publishes model, waits DS Platform
  • 33. DAIS 2021 33 Online Inference The Function - Serialized artifacted loaded on service instantiation, called during endpoints - Function shape used to create OpenAPI spec/validate inputs
  • 34. DAIS 2021 34 Online Inference The Context - Tag spec used to automatically deploy whenever new model is published - Note: user never has to call deploy()! Done through system-managed CD. - Stored package versions used to build docker images - Custom code made accessible to model for deserialization, execution Docker Image installed python packages custom code CD
  • 35. DAIS 2021 35 Online Inference The Data - Summary stats used to validate/monitor input (data drift) - Feature pointer used to load feature data Feature Store
  • 36. DAIS 2021 36 Batch Inference Approach Generate batch job in Stitch Fix workflow system (on top of airflow/flotilla) 1. Spins up spark cluster (if specified) 2. Loads input data, optionally joins with features 3. Execute model’s predict function over input 4. Saves to output table 1. Creates config for batch job (local/spark) a. tag query to choose model b. input/output tables 2. Executes as part of ETL DS Platform
  • 37. DAIS 2021 37 Batch Inference The function - Serialized artifacted loaded on batch job start - Function shape used to validate against inputs and outputs - MapPartitions + Pyarrow used to run models that take in DFs efficiently on spark -- abstracted away from user
  • 38. DAIS 2021 38 Batch Inference The context - Frozen package, language versions used in installing dependencies - Custom code made accessible to model for deserialization, execution - Tags used to determine which model to run Docker Image installed python packages custom code
  • 39. DAIS 2021 39 Batch Inference The data - Feature pointer used to load feature data if IDs specified - Evaluation table pointers stored in the registry Feature Store
  • 40. DAIS 2021 40 Metrics Tracking Approach Allow for metrics tracking with tag-based querying 1. Builds/manages dashboard 2. Adds fancy new metric types! 1. Logs metrics using python client 2. Explores in the Model Operations Dashboard 3. Saves URL for favorite viz DS Platform
  • 45. DAIS 2021 45 Value Added by Separating Concerns Making deployment easy Ensuring environment in prod == environment in training Providing easy metrics analysis Wrapping up complex systems Behind-the-scenes best practices Creating the best model Choosing the best libraries Determining the right metrics to log DS concerned with... Platform concerned with... DS focuses on creating the best model [writing the recipe] Platform focuses on optimal infrastructure [cooking it]
  • 47. DAIS 2021 47 Some Ideas... More advanced use of the data - production monitoring: utilize training data/stats to have visibility into prod/training drift More deployment contexts - Predictions on streaming/kafka topics More sophisticated feature tracking/integration - Feature stores are all the rage… Lambda-like architecture - Rather than requiring a deploy, can we query system for a model’s predictions? - Requires more unified environments… Attach external capabilities to replace home-built components of our own system...