Sparkling Water provides transparent integration of the H2O machine learning platform into the Spark ecosystem. It allows users to use advanced H2O machine learning algorithms like deep learning, gradient boosted machines, and random forests within existing Spark workflows. Sparkling Water excels at tasks that require these advanced algorithms, like complex predictive modeling problems. It also enables loading and parsing data directly into the H2O distributed in-memory framework using the H2OFrame data structure.

1. An introduction to Sparkling Water
Michal Malohlava
h2o.ai

2. Who Am I?
Background
• PhD in CS from Charles University in Prague,
Czech Republic
• Postdoc at Purdue University experimenting with
algos for large-scale computation
• Now at H2O.ai
Experience with domain-specific languages,
distributed system, software engineering,
and big data.

3. H2O.ai
H2Oteam
Sri Ambati Cliff Click
Co-Founders
Stephen
Boyd
Rob
Tibshirani
Trevor
Hastie
Scientific
Advisory
Council

4. H2O
Open-Source In-Memory Data Science Platform
• Highly optimized Java code (in-house)
• Distributed in-memory K-V store and map/
reduce computation framework
• Data parser (HDFS, S3, NFS, HTTP, local
drives, etc.)
• Read/write access to distributed data
frames (R/Pandas-style)
• ML algos - Deep Learning, GBM, DRF,
GLM, GLRM, K-Means, PCA, CoxPH,
Ensembles
• REST API: clients Interactive UI/R/Python
Sparkling
Water

5. Sparkling Water
Provides
• Transparent integration of H2O into Spark
ecosystem
• Use H2O Frames and algorithms with Spark API
Excels in existing Spark workflows requiring
advanced Machine Learning algorithms

6. Where to use Sparkling Water?
Data
Source
Model
building
Modelling
Deep Learning, GBM
DRF, GLM, GLRM
K-Means, PCA
CoxPH, Ensembles
Prediction
processingData munging

7. Where to use Sparkling Water?
Data
Source
Dataparsing
munging
Modelling
Data load/munging/
exploration
Load and parse
data directly into
H2OFrame
Ad hoc
data
transformation

8. Where to use Sparkling Water?
Data
Source
Off-line
model
training
Stream
processing
Data
Stream
Data munging
Model
prediction
Deploy
the model
Export model
in a binary format
or
as code
Modelling

9. WHAT IS INSIDE?

10. Cluster manager
Worker node
Spark executor
Scala/Py main
program
Driver node
H2OContext
SparkContext
Worker node
Spark executor
Worker node
Spark executor

11. H2OServicesH2OServices
Data
Source
SparkExecutorSparkExecutorSparkExecutor
Spark Cluster
DataFrame
H2OServices
H2OFrame
Data
Source
h2oContext.asDataFrame
h2oContext.asH2OFrame

12. TIME FOR DEMO

13. Key Points to Remember
Sparkling Water integrates H2O to Spark
• Enables using advanced
machine learning algorithms
inside Spark workflows
• Offers eager computation model,
mutable data structure
H2OFrame

14. THANK YOU.
@h2oai @mmalohlava
h2o.ai/download
github.com/h2oai/sparkling-water
Visit our booth for live demos and more!