You’ve fit your machine learning pipeline…now what? As a data scientist, taking a model to production is the single largest barrier to making an impact. As an engineer, how do you integrate machine learning into production applications? This talk will explore how we generalize the data science workflow through three stages: data collection, machine learning, and machine learning pipeline deployment. First, we’ll talk through how we leverage Spark Structured Streaming to generate consistent and up-to-date data that is available at training and scoring time. Next, we’ll discuss how we built repeatable, scalable, data agnostic machine learning pipelines that consider a host of algorithms, objective functions, feature selection and extraction methods to scale the impact of our data scientists. Finally, we’ll show you how to utilize MLeap to serialize these fitted Spark ML pipelines so they can be evaluated real-time, in tens of milliseconds.

1. Automating and Productionizing
Machine Learning Pipelines for
Real-Time Scoring
D a v i d C r e s p i , D a t a S c i e n t i s t
J a r e d P i e d t , S o f t w a r e E n g i n e e r

2. 2
What we do
• Digital consumer choice
platform
• Connect online
customers with products
and services across high-
growth industries:
• Home services
• Financial services
• Healthcare

3. 3
How does data science add value?

4. 4
Product recommendation

5. 5
Real-Time Predictions
Requirements
1
2
Speed
Consistency

6. 6
Data Science Process
1
2
3
Data Collection
Machine Learning Pipelines
Model Deployment

7. 7
Data Science Process
1
2
3
Data Collection
Machine Learning Pipelines
Model Deployment

8. 8
Old Data Architecture

9. 9
Old Data Architecture
D W
?

10. 10
Old Data Architecture
D W
Complex ETL

11. 11
Old Data Architecture
D W
Training
Data
Complex ETL

12. 12
Old Data Architecture
D W
A p p
Training
Data
Scoring
Data
Complex ETL

13. 13
Pain Points
• Duplication of business logic
• Data drift

14. 14
Goals
1
2
3
Immutable data
Write business logic once
Make data available in real-time

15. 15
Event-Driven Architecture
D a t a P i p e l i n e
W e b
C h a t
S e r v e r
I V R

16. 16
New Data Architecture
D a t a P i p e l i n e
Amazon
S3
B u s i n e s s
L o g i c
Training
Data
Scoring
Data

17. 17
Projections
{
i d : 4
…
} {
i d : 3
…
}
{
i d : 2
…
} {
i d : 1
…
}
{
i d :
…
}
Reducer

18. 18
Credit Card Recommendation
User Id Keyword Page View
Count
Card Shown Clicked
a best travel
cards
2 Travel 1
b credit cards 3 Cash Back 0
c top credit cards 1 Cash Back 1
d credit cards 1 Travel 0

19. 19
E1 E2 E3
time
r e d u c e r
D 1

20. 20
r e d u c e r
time
E1 E2 E3D 1

21. 21
r e d u c e r
time
E1 E2 E3D 1

22. 22
r e d u c e r
time
E1 E2 E3D 1

23. 23
Credit Card Recommendation
User Id Keyword Page View
Count
Card Shown Clicked
a best travel
cards
2 Travel 1
b credit cards 3 Cash Back 0
c top credit cards 1 Cash Back 1
d credit cards 1 Travel 0
z airline miles
card
1 Travel 1

24. 24
Data Science Process
1
2
3
Data Collection
Machine Learning Pipelines
Model Deployment

25. 25
ML Pipeline
Transformer Estimator

26. 26
Spark: Estimators and Transformers
Transformer
Estimator Transformer

27. 27
Spark: Estimators and Transformers
PipelineStage
Transformer
Estimator Transformer

28. 28
PipelineModel
Transformer Transformer Transformer
Pipeline
Transformer Transformer Estimator
PipelineModel
Transformer Transformer Transformer
Spark: Estimators and Transformers

29. 29
How do ML algorithms fit in?

30. 30
PipelineModel
Transformer Transformer Transformer
Pipeline
Transformer Transformer Estimator
PipelineModel
Transformer Transformer Transformer
Spark: Estimators and Transformers

31. 31
Generalizing Data Science
Response
All
Features
Response
Raw Text
Features
Categorical
Features
Numeric
Features
Training Data Training Data

32. 32
We fit our pipeline… now what?

33. 33
Data Science Process
1
2
3
Data Collection
Machine Learning Pipelines
Model Deployment

34. 34
Real-time scoring paradigm
? Prediction
API

35. 35
Model evaluation in real-time – with Spark

36. 36
Model evaluation in real-time – with MLeap
MLeap Runtime
Prediction API
MLeap Bundle

37. 37
Model deployment
Amazon
S3
MLeap Bundle

38. 38
MLeap Runtime
Prediction API
Prediction API
Amazon
S3

39. 39
Data collection
ML pipeline trainingModel deployment

40. 40
Recap
1
2
3
Data Collection
Machine Learning Pipelines
Model Deployment

41. 41
New capabilities
1>
New algorithms
300+
Real-Time
Scoring Models
# of people
required to
productionize
model

42. 42
We’re hiring!
• redventures.com/careers