The document discusses the role of data scientists and data engineers in creating smarter products, focusing on various deliverables like machine learning models, scripts, and dashboards. It outlines the machine learning training and serving pipeline using Apache Spark and addresses challenges related to model serving, including code duplication and integration issues. The text also mentions the need for a unified serving/scoring API and invites feedback, advisors, and early adopters for collaboration.

1. Spark Serving
by Stepan Pushkarev
CTO of Hydrosphere.io

2. Spark Users here?

3. Data Scientists and Spark Users here?

5. Why do companies hire data scientists?

6. Why do companies hire data scientists?
To make products smarter.

7. What is a deliverable of data scientist and data
engineer?

8. What is a deliverable of data scientist?
Academic
paper?
ML Model? R/Python
script?
Jupiter
Notebook?
BI
Dashboard?

9. cluster
data
model
data
scientist
? web
app

10. val wordCounts = textFile
.flatMap(line => line.split(" "))
.map(word => (word, 1))
.reduceByKey((a, b) => a + b)
executor
executorexecutor
executor executor

11. Machine Learning: training
+ serving

12. pipeline
Training (Estimation) pipeline
trainpreprocess preprocess

13. tokenizer
apache spark 1
hadoop mapreduce 0
spark machine learning 1
[apache, spark] 1
[hadoop, mapreduce] 0
[spark, machine, learning] 1

14. hashing tf
[apache, spark] 1
[hadoop, mapreduce] 0
[spark, machine, learning] 1
[105, 495], [1.0, 1.0] 1
[6, 638, 655], [1.0, 1.0, 1.0] 0
[105, 72, 852], [1.0, 1.0, 1.0] 1

15. logistic regression
[105, 495], [1.0, 1.0] 1
[6, 638, 655], [1.0, 1.0, 1.0] 0
[105, 72, 852], [1.0, 1.0, 1.0] 1
0 72 -2.7138781446090308
0 94 0.9042505436914775
0 105 3.0835670890496645
0 495 3.2071722417080766
0 722 0.9042505436914775

16. val tokenizer = new Tokenizer()
.setInputCol("text")
.setOutputCol("words")
val hashingTF = new HashingTF()
.setNumFeatures(1000)
.setInputCol(tokenizer.getOutputCol)
.setOutputCol("features")
val lr = new LogisticRegression()
.setMaxIter(10)
.setRegParam(0.001)
val pipeline = new Pipeline()
.setStages(Array(tokenizer, hashingTF, lr))
val model = pipeline.fit(training)
model.write.save("/tmp/spark-model")

17. pipeline
Prediction Pipeline
preprocess preprocess

18. val test = spark.createDataFrame(Seq(
("spark hadoop"),
("hadoop learning")
)).toDF("text")
val model = PipelineModel.load("/tmp/spark-
model")
model.transform(test).collect()

19. ./bin/spark-submit …

20. cluster
data
model
data
scientist
? web
app

21. Pipeline Serving - NOT Model Serving
Model level API leads to code duplication & inconsistency
at pre-processing stages!
Web App
Ruby/PHP:
preprocess
Check current user
User Logs
ML Pipeline: preprocess, train
Save
Score/serve model
Fraud Detection
Model

22. https://issues.apache.org/jira/browse/SPARK-16365
https://issues.apache.org/jira/browse/SPARK-13944

24. cluster
data
model
data
scientist
web
app
PMML
PFA
MLEAP
- Yet another Format Lock
- Code & state duplication
- Limited extensibility
- Inconsistency
- Extra moving parts

25. cluster
data
model
data
scientist
web
app
docker
model
libs
deps
- Fat All inclusive Docker - bad
practice
- Every model requires new
docker to be rebuilt

26. cluster
data
model
data
scientist
web
app
API
API
- Needs Spark Running
- High latency, low throughput

27. cluster
data
model
data
scientist
web
app
API
serving
API
+ Serving skips Spark
+ But re-uses ML algorithms
+ No new formats and APIs
+ Low Latency but not super tuned
+ Scalable

28. Low level API Challenge
MS Azure

29. A deliverable for ML model
Single row Serving / Scoring
layer
xml, json, parquet, pojo, other
Monitoring,
testing
integration
Large Scale,
Batch
processing
engine

30. Zooming out
Unified Serving/Scoring API
Repository
MLLib model TensorFlow model Other model

31. Real-time Prediction PIpelines

32. Starting from scratch - System ML
Multiple execution modes, including Spark MLContext
API, Spark Batch, Hadoop Batch, Standalone, and JMLC.

33. Demo Time

34. Thank you
Looking for
- Feedback
- Advisors, mentors & partners
- Pilots and early adopters
Stay in touch
- @hydrospheredata
- https://github.com/Hydrospheredata
- http://hydrosphere.io/
- spushkarev@hydrosphere.io