Production Model Deployment - StitchFix - 2018
•
1 like•201 views
Stitch Fix Model Deployment
Recommended
Recommended
More Related Content
Similar to Production Model Deployment - StitchFix - 2018
Similar to Production Model Deployment - StitchFix - 2018 (20)
More from Karthik Murugesan
More from Karthik Murugesan (20)
Recently uploaded
Recently uploaded (20)
Production Model Deployment - StitchFix - 2018
- 2. @j_houg 2 whoami And other, failed startups
- 3. Agenda @j_houg 3 MODEL LIFECYCLE DEPLOYMENT CHALLENGES SOLUTIONS THAT THEMSELVES ARE CHALLENGES THERE ARE SOLUTIONS TO THOSE TOO! THEY ARE ALSO CHALLENGING
- 5. @j_houg Black Box Data Scientists
- 6. @j_houg Black Box Data Scientists
- 8. @j_houg 8 Data Warehouse Featurization Training ValidationApplication
- 9. @j_houg 9 Deploying a Model
- 11. @j_houg 11 3 Modes of Deployment Data Warehouse
- 12. @j_houg 12 What is a model? Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction
- 14. @j_houg 14 Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction Serialized Model Server
- 16. @j_houg 16 3 Questions Does your model do what you need? Does it meet your engineering requirements? Is your team organized to build and support it?
- 18. @j_houg 18 Useful? George Box (kind of) said “All models are wrong, some are useful.”
- 20. @j_houg 20 Continues to be useful? Forever? Distribution of features Distribution of predictions
- 21. @j_houg 21 Continues to be useful? Forever? Distribution of features Distribution of predictions
- 22. @j_houg 22 Pipelines We don’t deploy one model, we deploy the process for repeatedly making more
- 23. @j_houg 23 Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction Serialized Model Featurization Training ValidationApplication Serialized Model
- 29. @j_houg Lambda & Throughput? oryx.io and https://www2007.org/papers/paper570.pdf
- 31. @j_houg 31 Fast enough? Materialize model features Application New Observation Featurization Learned Prediction Feature Store
- 32. @j_houg 32 Fast enough? Materialize model output Application New Observation Featurization Learned Prediction Model Output
- 37. @j_houg 37 Conway’s Law "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations." — M. Conway
- 38. @j_houg Typical Data Science Department Data Engineers Data Scientists Software Engineers Data Infrastructure Engineers provide resources & support
- 39. @j_houg Typical Data Science Department Data Driven Capability Data Engineers Data Scientists Software Engineers Infrastructure Engineers
- 40. @j_houg Lambda & Team Structure oryx.io and https://www2007.org/papers/paper570.pdf
- 41. @j_houg 41 Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction Serialized Model
- 42. @j_houg 42 Fix Model Handoff? #WOCTechChat
- 43. @j_houg Capabilities Data Platform Engineers specialized by function Capability A Capability B Capability C Capability D DataScientists specialized by capability DataScientists DataScientists DataScientists
- 44. @j_houg 44 Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction Serialized Model
- 46. @j_houg 46 Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction Serialized Model Open Scoring
- 48. @j_houg 48 Data Warehouse Featurization Training Validation Application New Observation Featurization Application Learned Prediction Serialized Model
- 49. @j_houg 49 Does the model… Do the thing you want it to? - Functionality - Usefulness - Both continually Meet your requirements? - Throughput - Latency - Freshness
- 50. @j_houg 50 Is your team… Organized in a way that supports your system and its requirements?
- 51. @j_houg Thanks!