The document discusses the process of deploying machine learning models in the cloud, focusing on the experiences of a character named Bob who leads a team of data scientists. It outlines the challenges and strategies of deployment, including the use of web-apps, servers, auto-scaling, and serverless options. Finally, it highlights the advantages of serverless architecture, such as scalability and reduced operational concerns.

1. How to deploy Machine Learning
models in the Cloud
Alex Casalboni
PYCON SETTE
@ Firenze
clda.co/pycon7-alex

2. About Me
PYCON SETTE @ FIRENZE
@alex_casalboni
clda.co/pycon7-alex
Bachelor in Computer Science
Master in Sound & Music Engineering
6+ years of Web Development
3+ years of Python

3. Have you ever met a Data Scien@st?
PYCON SETTE @ FIRENZE
Bob
Bob leads a skilled team of data scienCsts
Bob has built a great ML model
Bob loves Python
Bob likes user-­‐centered ML
clda.co/pycon7-alex

4. How did Bob build his model in Python?
PYCON SETTE @ FIRENZE
clda.co/pycon7-code
clda.co/pycon7-alex
Bob

5. PYCON SETTE @ FIRENZE
GPU availability
distributed compuCng
infrastructure-­‐aware code
1. Bob needs Elas@city
clda.co/pycon7-alex
And why is deploying a challenge?
Bob

6. PYCON SETTE @ FIRENZE
MulC-­‐model architecture
A/B tesCng & versioning
RESTful API
2. Bob needs plenty of Features
Auth layer
clda.co/pycon7-alex
And why is deploying a challenge?
Bob

7. PYCON SETTE @ FIRENZE
infrastructure maintenance
conCnuous delivery & deployments
automaCon, performance & monitoring
3. Data Scien@sts ain’t DevOps
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data
ScienCst
DevOps

8. PYCON SETTE @ FIRENZE
model design, training & tesCng
code reviews & debugging
data exploraCon
4. DevOps ain’t Data Scien@sts
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data
ScienCst
DevOps

9. Deployment Strategies
PYCON SETTE @ FIRENZE
1. Web-­‐app controller
clda.co/pycon7-alex
how will Bob update his model(s)?
same website auth layer?
shared (limited) uWSGI processes?
It’s disarmingly simple, but…
Bob

10. Deployment Strategies
PYCON SETTE @ FIRENZE
2. Fleet of servers
clda.co/pycon7-alex
same problems as before
Bob has many more machines to maintain
sCll no elasCcity (over-­‐provisioning)
Bigger capacity and no code changes, but…
Bob Bob2 Bob3
Load Balancing

11. Deployment Strategies
PYCON SETTE @ FIRENZE
3. Auto Scaling
clda.co/pycon7-alex
<Bob>
sCll shared resources? containers?
even bigger lack of ownership
what about caching, versioning and auth?
Bob achieved elasCcity and availability , but…
AWS ELB + Auto Scaling
(or maybe ElasCc Beanstalk?)

12. Deployment Strategies
PYCON SETTE @ FIRENZE
4. Serverless
clda.co/pycon7-alex
+
Amazon API
Gateway
AWS
Lambda
Happy
Bob
Infinite scalability, global CDN and caching
Transparent elasCcity (and PAYG)
RESTful & auth layer + versioning (for free!)
Bob doesn’t need to be a DevOps anymore!

13. The Serverless Cloud
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
There is always a server somewhere,
you just don't have to worry about it :)

14. Does Bob’s code change a lot?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Basically, no change!
Bob

15. How does Bob keep the model updated?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Bob
Hint: use gzip!

16. PYCON SETTE @ FIRENZE
clda.co/pycon7-lambda
clda.co/pycon7-alex

17. Serverless Cloud Domina@on
PYCON SETTE @ FIRENZE
Amazon Web Services (Lambda)
Microso` Azure (FuncCons)
Google Cloud Plaborm (FuncCons)
IBM Bluemix (OpenWhisk)
hook.io
Cloud PlaSorms Hybrid & Microservices
stackhut.com
webtask.io
iron.io
clda.co/pycon7-alex

18. Thank you :)
jobs.cloudacademy.com
P.S. we are hiring!