Sparklyr has enabled data scientists to use familiar R and tidyverse syntax to interactively analyze data and build models at scale via Apache Spark. However, one common pain point in organizations is operationalizing these models either in a batch prediction or real-time scoring setting. With support for Spark ML pipelines in sparklyr, data scientists can use R to build pipelines that are fully interoperable with Scala using a familiar API. For real-time scoring, an R interface to MLeap, an open source engine for serializing and serving Spark ML models, is provided. These functionalities faciliate collaboration among data scientists and implementation engineers and shorten time to production. We discuss the mechanics of sparklyr ML pipelines and demonstrate an end-to-end example.

1. From Prototyping to Deployment at Scale with R and sparklyr
Kevin Kuo
June 2018
1 / 26

2. Menu today
The deployment problem
ML pipelines
Model deployment and demo
2 / 26

3. The deployment problem
3 / 26

4. Deployment
Or, putting ML models "into production".
4 / 26

5. Deployment
Or, putting ML models "into production".
Basically, make it so that someone else can use your model (i.e. make some predictions with
it).
4 / 26

6. Batch
Event-based/time-based
E.g. nightly portfolio risk calculations,
and
Email campaigns
It's OK to take a while
"Real-time"
On demand
E.g. instant loan approvals, and
Fraud detection on credit card swipes
Gotta be (relatively) fast, seconds to
less than a second
Deployment - latency dimension
5 / 26

7. Deployment - why is it hard?
Challenge 1/n: Putting ML models into production involves different
expertise
6 / 26

8. Deployment - why is it hard?
Challenge 1/n: Putting ML models into production involves different
expertise
...so it involves more people
6 / 26

9. Deployment - why is it hard?
Challenge 1/n: Putting ML models into production involves different
expertise
...so it involves more people
...mo ppl mo problems
6 / 26

10. Deployment - mo ppl mo problems
Credit: https://youtu.be/-K9SjrWpeys @josh_wills 7 / 26

11. Deployment - why is it hard?
Challenge 2/n: Rapidly changing landscape in deployment options
8 / 26

12. Deployment - why is it hard?
Challenge 2/n: Rapidly changing landscape in deployment options
What do?
Spark ML persistence?
dbml-local?
PMML?
PFA/Aardpfark?
MLeap?
ONNX?
Roll our own thing?
Re-implement the model in C++, because performance?
Throw it into a container and do orchestration cuz it's cool?
8 / 26

13. Deployment - why is it hard?
Challenge 3/n: Too many ML frameworks and no standardization
9 / 26

14. Deployment - why is it hard?
Challenge 3/n: Too many ML frameworks and no standardization
We're focusing on Spark in this session, but we'll acknowledge other technologies we need to deal with
9 / 26

15. Deployment - diversity of ML frameworks
Spark ML, xgboost, random CRAN packages, scikit-learn, H2O, ...
10 / 26

16. Deployment - one of many scenarios
Data scientist: Hey this random forest loan decision model is ready to go!
Engineer: OK, we need to recode it in C#/Java, see you in 6 months!
11 / 26

17. Deployment - one of many scenarios
Data scientist: Hey this random forest loan decision model is ready to go!
Engineer: OK, we need to recode it in C#/Java, see you in 6 months!
Data scientist: Oh no that's too long, what about I give you this GLM with just a few
parameters?
Engineer: 2 months.
11 / 26

18. Deployment - one of many scenarios
Data scientist: Hey this random forest loan decision model is ready to go!
Engineer: OK, we need to recode it in C#/Java, see you in 6 months!
Data scientist: Oh no that's too long, what about I give you this GLM with just a few
parameters?
Engineer: 2 months.
Data scientist:
11 / 26

19. Deployment - challenges for the R user
On average, R users tend to...
Be math/stats types
Have little CS/software engineering training
So it's slightly tougher for them to collaborate with the folks doing model implementation.
12 / 26

20. Deployment - challenges for the R user
On average, R users tend to...
Be math/stats types
Have little CS/software engineering training
So it's slightly tougher for them to collaborate with the folks doing model implementation.
However,
Data scientists (regardless of background) are becoming more comfortable moving up and down the stack
There has been active development of technology to faciliate the data science-engineering handoff
12 / 26

21. Deployment - technology
Technology won't solve your people/process/culture issues, but it can make collaboration easier!
13 / 26

22. Deployment - technology
Technology won't solve your people/process/culture issues, but it can make collaboration easier!
Next up: we'll provide a quick review of Spark ML pipelines, and offer a couple ways of
"deploying" them using the sparklyr ecosystem.
13 / 26

23. Spark ML pipelines
14 / 26

24. ML pipelines
Pipelines are basically...
A structure in which you can throw in data transformers and ML models.
15 / 26

25. ML pipelines
Pipelines are basically...
A structure in which you can throw in data transformers and ML models.
Keep in mind that when you deploy a model, you also need to deploy the feature engineering steps in order to
feed the right inputs to the model! (E.g. converting a numeric age variable into an age range bucket that the
model requires.)
15 / 26

26. ML pipelines
Pipelines are basically...
A structure in which you can throw in data transformers and ML models.
Keep in mind that when you deploy a model, you also need to deploy the feature engineering steps in order to
feed the right inputs to the model! (E.g. converting a numeric age variable into an age range bucket that the
model requires.)
Now let's go through a (very quick) overview of pipeline concepts.
15 / 26

27. ML pipelines
A Transformer takes a data frame, via ml_transform(), and returns a transformed data frame.
16 / 26

28. ML pipelines
A Transformer takes a data frame, via ml_transform(), and returns a transformed data frame.
An Estimator take a data frame, via ml_fit(), and returns a Transformer.
17 / 26

29. ML pipelines
A Transformer takes a data frame, via ml_transform(), and returns a transformed data frame.
An Estimator take a data frame, via ml_fit(), and returns a Transformer.
A Pipeline consists of a sequence of stages—PipelineStages—that act on some data in order.
A PipelineStage can be either a Transformer or an Estimator.
18 / 26

30. ML pipelines
A Pipeline consists of a sequence of stages—PipelineStages—that act on some data in order.
A PipelineStage can be either a Transformer or an Estimator.
A Pipeline is always an Estimator, and its fitted form is called PipelineModel which is a
Transformer.
19 / 26

31. ML pipelines
20 / 26

32. ML pipelines
21 / 26

33. Serving the model
Now, the trick is to persist this PipelineModel so we can use it to serve predictions later on.
22 / 26

34. Serving the model
Now, the trick is to persist this PipelineModel so we can use it to serve predictions later on.
We'll demo a couple ways today
Native Spark ML persistence support
MLeap (via the mleap R package)
22 / 26

35. Demo
23 / 26

36. Spark ML Persistence
Appropriate for batch jobs, scoring lots
of records at once
Requires Spark session
MLeap
Better for real-time prediction of a
small number of records
Doesn't require Spark session, portable
to apps/devices that support JVM
Model deployment paths
24 / 26

37. Towards a better deployment story
Data scientist: Hey this random forest loan decision model is ready to go! Here is the .zip
bundle, and here is the documentation you need to use it.
Engineer: Awesome! We won't need to write a bazillion if-else statements to recreate the
model!
25 / 26

38. Towards a better deployment story
Data scientist: Hey this random forest loan decision model is ready to go! Here is the .zip
bundle, and here is the documentation you need to use it.
Engineer: Awesome! We won't need to write a bazillion if-else statements to recreate the
model!
When the model needs updating...
25 / 26

39. Towards a better deployment story
Data scientist: Hey this random forest loan decision model is ready to go! Here is the .zip
bundle, and here is the documentation you need to use it.
Engineer: Awesome! We won't need to write a bazillion if-else statements to recreate the
model!
When the model needs updating...
Data scientist: We decided to use a GBM instead for better accuracy, here's the updated
bundle.
Engineer: Fantabulous! All we need to do is update the model directory!
25 / 26

40. Wrap up
Slides and code will be available at https://kevinykuo.com.
Inspirations/other talks to check out
"Productionizing Spark ML pipelines with the portable format for analytics" https://youtu.be/h-
B0VCkoRkE @MLnick
"How to Productionize Your Machine Learning Models Using Apache Spark MLlib 2.x"
https://youtu.be/r740xbIpb54 Richard Garris
"MLeap and Combust ML" https://youtu.be/MGZDF6E41r4 Hollin Wilkins and Mikhail Semeniuk
26 / 26