DataScience Lab, 13 мая 2017 Cервинг моделей, построенных на больших данных с помощью Apache Spark Степан Пушкарев (GM (Kazan) at Provectus / CTO at Hydrosphere.io) После подготовки данных и обучения моделей на больших данных с использованием Apache Spark встает вопрос о том, как использовать обученные модели в реальных приложениях. Помимо модели важно не забывать про весь пайплайн пре-процессинга данных, который должен попасть в продакшн в том виде, в котором его спроектировал и реализовал дата саентист. Такие решения, как PMML/PFA, основанные на экспорте/импорте модели и алгоритма имеют очевидные недостатки и ограничения. В данном докладе мы предложим альтернативное решение, которое упрощает процесс использования моделей и пайплайнов в реальных боевых приложениях. Все материалы доступны по ссылке: http://datascience.in.ua/report2017

1. Spark Serving
by Stepan Pushkarev
CTO of Hydrosphere.io

2. Spark Users here?

3. Data Scientists and Spark Users here?

5. Why do companies hire data scientists?

6. Why do companies hire data scientists?
To make products smarter.

7. What is a deliverable of data scientist and data
engineer?

8. What is a deliverable of data scientist?
Academic
paper?
ML Model? R/Python
script?
Jupiter
Notebook?
BI
Dashboard?

9. cluster
data
model
data
scientist
? web
app

10. val wordCounts = textFile
.flatMap(line => line.split(" "))
.map(word => (word, 1))
.reduceByKey((a, b) => a + b)
executor
executorexecutor
executor executor

11. Machine Learning: training
+ serving

12. pipeline
Training (Estimation) pipeline
trainpreprocess preprocess

13. tokenizer
apache spark 1
hadoop mapreduce 0
spark machine learning 1
[apache, spark] 1
[hadoop, mapreduce] 0
[spark, machine, learning] 1

14. hashing tf
[apache, spark] 1
[hadoop, mapreduce] 0
[spark, machine, learning] 1
[105, 495], [1.0, 1.0] 1
[6, 638, 655], [1.0, 1.0, 1.0] 0
[105, 72, 852], [1.0, 1.0, 1.0] 1

15. logistic regression
[105, 495], [1.0, 1.0] 1
[6, 638, 655], [1.0, 1.0, 1.0] 0
[105, 72, 852], [1.0, 1.0, 1.0] 1
0 72 -2.7138781446090308
0 94 0.9042505436914775
0 105 3.0835670890496645
0 495 3.2071722417080766
0 722 0.9042505436914775

16. val tokenizer = new Tokenizer()
.setInputCol("text")
.setOutputCol("words")
val hashingTF = new HashingTF()
.setNumFeatures(1000)
.setInputCol(tokenizer.getOutputCol)
.setOutputCol("features")
val lr = new LogisticRegression()
.setMaxIter(10)
.setRegParam(0.001)
val pipeline = new Pipeline()
.setStages(Array(tokenizer, hashingTF, lr))
val model = pipeline.fit(training)
model.write.save("/tmp/spark-model")

17. pipeline
Prediction Pipeline
preprocess preprocess

18. val test = spark.createDataFrame(Seq(
("spark hadoop"),
("hadoop learning")
)).toDF("text")
val model = PipelineModel.load("/tmp/spark-
model")
model.transform(test).collect()

19. ./bin/spark-submit …

20. cluster
data
model
data
scientist
? web
app

21. Pipeline Serving - NOT Model Serving
Model level API leads to code duplication & inconsistency
at pre-processing stages!
Web App
Ruby/PHP:
preprocess
Check current user
User Logs
ML Pipeline: preprocess, train
Save
Score/serve model
Fraud Detection
Model

22. https://issues.apache.org/jira/browse/SPARK-16365
https://issues.apache.org/jira/browse/SPARK-13944

24. cluster
data
model
data
scientist
web
app
PMML
PFA
MLEAP
- Yet another Format Lock
- Code & state duplication
- Limited extensibility
- Inconsistency
- Extra moving parts

25. cluster
data
model
data
scientist
web
app
docker
model
libs
deps
- Fat All inclusive Docker - bad
practice
- Every model requires new
docker to be rebuilt

26. cluster
data
model
data
scientist
web
app
API
API
- Needs Spark Running
- High latency, low throughput

27. cluster
data
model
data
scientist
web
app
API
serving
API
+ Serving skips Spark
+ But re-uses ML algorithms
+ No new formats and APIs
+ Low Latency but not super tuned
+ Scalable

28. Low level API Challenge
MS Azure

29. A deliverable for ML model
Single row Serving / Scoring
layer
xml, json, parquet, pojo, other
Monitoring,
testing
integration
Large Scale,
Batch
processing
engine

30. Zooming out
Unified Serving/Scoring API
Repository
MLLib model TensorFlow model Other model

31. Real-time Prediction PIpelines

32. Starting from scratch - System ML
Multiple execution modes, including Spark MLContext
API, Spark Batch, Hadoop Batch, Standalone, and JMLC.

33. Demo Time

34. Thank you
Looking for
- Feedback
- Advisors, mentors & partners
- Pilots and early adopters
Stay in touch
- @hydrospheredata
- https://github.com/Hydrospheredata
- http://hydrosphere.io/
- spushkarev@hydrosphere.io