Machine Learning at Scale. Kubernetes Meetup 2019 SG. Discuss the chasm between developing ML models to deployment into production using Kubernetes.

1. Machine Learning at Scale
Speaker: Wesley GOI
17 Jan 2019

2. About me
Wesley GOI
• Data Scientist @honestbee 🐝
• Computational Biology @NUS

3. @School
SGE Cluster

4. now….
got to be kidding me
@ home
@ work

5. Academic Setting
Industry

6. Day 1 Day 2
My Kubernetes Journey
October 2018

8. credits @joerg_schad
What I want to be doing

9. credits @joerg_schad

10. Division of LabourData Ops / ML Engineer / AI OPs
credits @joerg_schad

11. 1. Analysis Tools

12. Data Scientists
Pod
Data science image
• Batteries included
• Bare minimum knowledge of linux OS
Kubespawner
https://zero-to-jupyterhub.readthedocs.io/en/latest/
https://github.com/jupyterhub/zero-to-jupyterhub-k8s
CPU
GPU

13. Step 2: Login
Step 1: Port Forward
Step 4: Rock and RollStep 3:
• Choose Image
• GPU / CPU / Memory

14. 2. ML Pipelines

15. ML Pipelines
Useful for
1. Parameter search
2. Retraining
ML model

16. ML Pipelines
Useful for
1. Parameter search
2. Retraining
Feature Extraction
Feature Processing
Training
Evaluation
DAG / Workflow

17. Job submission
SGE Cluster
qsub -t 1-10 -v PATH=$PATH -v R_LIBS=$R_LIBS `pwd`/example.R <hyperparameters>
CPU cores required script

18. Job submission

19. Job submission (K8S)
https://argoproj.github.io/

20. Argos
Requirements
• argo namespace
• UI pod
• Workflow controller pod

21. argo (re)submit /path/to/workflow.yaml
<YAML>
argo get <job_name>-<id> argo logs -w <job_name>-<id>

22. apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: steps-
spec:
entrypoint: hello-hello-hello
# This spec contains two templates: hello-hello-hello and whalesay
templates:
- name: hello-hello-hello
# Instead of just running a container
# This template has a sequence of steps
steps:
- - name: hello1 #hello1 is run before the following steps
template: whalesay
arguments:
parameters:
- name: message
value: "hello1"
- - name: hello2a #double dash => run after previous step
template: whalesay
arguments:
parameters:
- name: message
value: "hello2a"
- name: hello2b #single dash => run in parallel with previous step
template: whalesay
arguments:
parameters:
- name: message
value: "hello2b"
# This is the same template as from the previous example
- name: whalesay
inputs:
parameters:
- name: message
container:
image: docker/whalesay
command: [cowsay]
args: ["{{inputs.parameters.message}}"]
Parallel
Sequential
<Name of workflow>-<id>

23. Deployment

24. Deployment

25. https://www.youtube.com/watch?v=huqpXMNFD54
credits: EmilyGorenski

26. DataScientist
Feature
µservices
Precalculated Results
Experimentation Framework
Backend
Stage 1
Stage 2
DataOPs
Boilerplate helm deployments allows for rapid deployment of models (Good solution is still not found because every deployment is unique)
• solutions
seldon
• pipeline.ai

27. Not complete many more steps to go!

28. Thank you