This document discusses Cloudera's Data Science Workbench (CDSW) product. It begins with an introduction and agenda. It then discusses challenges with data science projects and how CDSW aims to help by providing a shared platform for data access, analytics and model deployment. The document outlines CDSW's architecture built on Docker and Kubernetes. It demonstrates CDSW's capabilities and integrations with Cloudera's Data Hub platform before concluding with information about Cloudera's research team.

1. 1© Cloudera, Inc. All rights reserved.
Cloudera Data Science Workbench
Arun Pamulapati, Senior Systems Engineer
Cloudera Certified Developer and Administrator
Rich Stevens, Customer Account Manager

2. 2© Cloudera, Inc. All rights reserved.
● The information in this document is proprietary to Cloudera. No part of this document may be
reproduced, copied or transmitted in any form for any purpose without the express prior written
permission of Cloudera.
● This document is a preliminary version and not subject to your license agreement or any other
agreement with Cloudera. This document contains only intended strategies, developments and
functionalities of Cloudera products and is not intended to be binding upon Cloudera to any particular
course of business, product strategy and/or development. Please note that this document is subject to
change and may be changed by Cloudera at any time without notice.
● Cloudera assumes no responsibility for errors or omissions in this document. Cloudera does not
warrant the accuracy or completeness of the information, text, graphics, links or other items contained
within this material. This document is provided without a warranty of any kind, either express or
implied, including but not limited to the implied warranties of merchantability, fitness for a particular
purpose or non-infringement.
● Cloudera shall have no liability for damages of any kind including without limitation direct, special,
indirect or consequential damages that may result from the use of these materials. The limitation shall
not apply in cases of gross negligence.
Disclaimer

3. 3© Cloudera, Inc. All rights reserved.
● Introduction
● The state of the industry
● What it takes to succeed
● CDSW Architecture
● CDSW Demo
● CDH Stack
● Q&A
Agenda

4. 4© Cloudera, Inc. All rights reserved.
Getting to know you …
How many of you have used any Hadoop projects before?
Used Hive or Impala / SQL on Hadoop?
Deployed one or more Hadoop applications into production?
How many of you would describe yourselves as a Data Scientist?
Deployed one or more Hadoop applications into sandbox environment?
What languages do you use?
• Python
• R/sparklyr
• Scala
• SparkMLlib
• H2O
• TensorFlow

5. 5© Cloudera, Inc. All rights reserved.
Cost of compute
Data volume
Time
1950
s
1960
s
1970
s
1980
s
1990
s
2000
s
2010
s
Why machine learning is a hot topic today.

6. 6© Cloudera, Inc. All rights reserved.
We are in the age of machine learning
Data has never been
more plentiful
Open source data science and
machine learning libraries are
rapidly evolving
Flexible commodity storage
and compute make scalable
production machine learning
affordable
Data Analytics Deployment

7. 7© Cloudera, Inc. All rights reserved.
Driving Board-Level Value
Drive Customer
Insights/Know your
Customers
Lower Business
Risk/ Secure your
Business
• Market Segmentation
• Customer 360
• Next Best Offer
• Churn Analysis
• Click-Stream
• Risk Modeling & Analysis
• Network Threat
Detection
• Credit Risk Assessment
• SPAM Detection

8. 8© Cloudera, Inc. All rights reserved.
But there are practical challenges
Most data science done
at small scale, individually,
and is difficult to replicate
Very few models
reach production
Teams have different,
conflicting requests for
languages & libraries
Data needs to move
across multiple
different systems
Data Analytics Deployment

9. 9© Cloudera, Inc. All rights reserved.
Help more data scientists
use the power of Cloudera
Use a powerful, familiar
environment with direct access
to Cloudera data and compute
Data Scientist
Data Engineer
Make it easy and secure to
add new users, use cases
Offer secure self-service
analytics and a faster path to
production on common,
affordable infrastructure
Enterprise Architect
Hadoop Admin
Our goal: Open data science at enterprise scale

10. 10© Cloudera, Inc. All rights reserved.
Shared: Data, Operations, Governance, Security, Metadata
Data Engineering Data Science Deployment
Data Wrangling
Visualization
and Analysis
Model Training
& Testing Batch Scoring
Online Scoring
Serving
Data GovernanceCuration
Processing
Acquisition
Reports, Dashboards
Dev: Collaboration, Version Control Ops: Deployment, Scheduling, Orchestration
Support the complete data science workflow
From data to exploration to action

11. 11© Cloudera, Inc. All rights reserved.
Data science at enterprise scale requires a full stack
• Support unlimited data
• Provide sufficient tools for Analysts
• Provide sufficient tools for
Data Scientists + Data Engineers
• Enable real-time use cases
• Provide data governance
• Provide full-stack security
• Deploy in the cloud
• Integrate with partner tools
• Be easy for IT to deploy/maintain
• Accelerate your data and machine
learning strategy with expert research
and advising
✓Hadoop
✓Impala, Hive, Hue
✓Spark, Data Science Workbench
✓Kafka, Spark Streaming
✓Navigator + Partners
✓Kerberos, Sentry, Record Service, KMS/KTS
✓Cloudera Director/ Altus (PaaS)
✓Rich Ecosystem
✓Cloudera Manager + Director
✓Cloudera Fast Forward Labs

12. 12© Cloudera, Inc. All rights reserved.
Accelerates data science from
development to production with:
●Secure self-service data access
●On-demand compute
●Support for Python, R, and Scala
●Project dependency isolation for
multiple library versions
●Workflow automation, version
control, collaboration and sharing
Cloudera Data Science Workbench
Self-service data science for the enterprise

13. 13© Cloudera, Inc. All rights reserved.
A modern data science architecture
CDH CDH
Cloudera Manager
gateway nodes CDH nodes
●Built on Docker and Kubernetes
●Runs on dedicated gateway nodes
●User sessions run in isolated
“engine” containers which:
○Host Kerberos-authenticated
Python/R/Scala runtimes
○Interact with Spark via YARN
client mode (Driver runs in
container, workers on CDH)
Hive, HDFS, ...
CDSW CDSW
...
Master
...
Engine
EngineEngine
EngineEngine

14. 14© Cloudera, Inc. All rights reserved.
Demo

15. 15© Cloudera, Inc. All rights reserved.
CDH platform
OPERATIONS
Cloudera Manager
Cloudera Director
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee
Optimizer
STRUCTURED
Sqoop
UNSTRUCTURED
Kafka, Flume
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
STORE
INTEGRATE
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
OTHER
Kite
NoSQL
HBase
OTHER
Object Store
FILESYSTEM
HDFS
RELATIONAL
Kudu

16. 16© Cloudera, Inc. All rights reserved.
Cloudera Fast Forward Labs
Scientists and engineers Led by Hilary Mason (Fortune 40 under 40)
Our team blends research fluency with engineering skill and experience
building products at top technologycompanies.

17. 17© Cloudera, Inc. All rights reserved.
Thank you
Arun Pamulapati
apamulapati@cloudera.com
678.524.3778
https://github.com/pamulapati/cdswdemo