Successfully reported this slideshow.
Leverage: Multi-In + Scale + Multi Out with Cloudera as Hadoop platform
Ingest Flume Kafka Sqoop Spark Datascience Workbench
PaaS Altus job-first
Data Governance Lineage Security GDPR Navigator
1© Cloudera, Inc. All rights reserved.
Speedpitch @ TDWI
Big Data Integration
2© Cloudera, Inc. All rights reserved.
Cloudera - company snapshot
Founded 2008, by former employees of
Funding More than $1B invested, $740M primary investment from
NOW Publicly Traded on the NYSE: CLDR
Employees Today 1,500+ worldwide
World Class Support Pro-active & predictive support programs using our EDH
Mission Critical Production deployments in run-the-business applications worldwide
– Financial Services, Pharma, Retail, Telecom, Media, Health Care,
Largest Ecosystem More than 2,600 Partners
Cloudera University Over 40,000 trained
Open Source Leaders Cloudera employees are leading developers & contributors to the
complete Apache Hadoop ecosystem of projects
3© Cloudera, Inc. All rights reserved.
4© Cloudera, Inc. All rights reserved.
LEGACY = Data to Compute MODERN = Compute to Data
businesses use all data:
internal & external data
of all types
Structured data mainly
Internal data only
“Important” data only
Siloed data sources
The “paradigm shift” to Hadoop / data centric platforms
5© Cloudera, Inc. All rights reserved.
Big Data Technology = Multi-In + Scale + Multi-Out
1. Multi-In: Process different types of data together
Structured: From relational and transactional systems (RDBMS).
Semi-structured: e.g. Server Logs, Sensor Logs, Clickstreams, …
Unstructured: e.g. Emails, Tweets, Images, Audio, Video, …
2. Scale technically & economically (reduce
3. Multi-Out: Run different types of data processing
workloads as part of a unified data pipeline.
©2014 Cloudera, Inc. All rights reserved.
6© Cloudera, Inc. All rights reserved.
The Cloudera data management platform
Data Sources Data Ingest Data Storage & Processing
Serving, Analytics &
Stream or batch ingestion of IoT data
Ingestion of data from relational sources
Storage (HDFS) & Batch (HIVE)
Storage & serving for fast changing data
NoSQL data store for real-time apps
MPP SQL for fast analytics
Real time searchConnected Things/
Security, Scalability & Easy Management
Flexibility: Datacenter Cloud
Stream & iterative processing, ML
7© Cloudera, Inc. All rights reserved.
Log & EventAggregation for Hadoop
• Efficiently move large amounts
of streaming/log data
• Easily collect data from multiple
• Built-in sources, sinks, and
• Customize data flow to transform
• Reliable, scalable, and
extensible for production
• Manage and monitor with
Social Media Posts
8© Cloudera, Inc. All rights reserved.
Pub-Sub Messaging for Hadoop • Backbone for real-time architectures
• Fast, flexible messaging for a wide
range of use cases
• Scale to support more data sources and
growing data volumes
• Zero data loss durability and always-on
• Built-in security and data protection
• Seamless integration across the
• Connect to Flume, Spark Streaming,
HBase, and more
• Manage and monitor with Cloudera
Kafka decouples Data Pipelines
9© Cloudera, Inc. All rights reserved.
SQL to Hadoop
• Efficiently exchange data between database and Hadoop
• Import all or partial/new data
• Export for shared data access across systems
• Easily get started with high performance connectors
• Free to use
• Optimized connectors for popular RDBMS, EDW, and NoSQL options
Database Hadoop Cluster
10© Cloudera, Inc. All rights reserved.
Go beyond SQL with Python & Spark:
Cloudera Data Science Workbench
Accelerates data engineering from
development to production with:
• Secure self-service environments
for data scientists to work against
• Support for Python, R, and Scala,
plus project dependency isolation
for multiple library versions
• Workflow automation, version
control, collaboration and sharing
11© Cloudera, Inc. All rights reserved.
Cloudera Altus PaaS for Data Engineering
Platform as a service for ETL
(machine learning, and data
● Pay as you Go
● Support for MR2, Hive, Spark,
● Job-first orientation
● Quick and easy workload
troubleshooting & analytics
12© Cloudera, Inc. All rights reserved.
DI/DQ/Profiling/Wrangling solutions from partners
13© Cloudera, Inc. All rights reserved.
Data stewardship and governance solutions
Centralized Stewardship End User Discovery
Unified technical metadata catalog
Extensible business metadata and glossary
Metadata rules engine
Unified audit/access logs
Dashboards and analytics
APIs for augmentation and consumption
End user collaboration
Enterprise aggregation: metadata, lineage, SIEM,
14© Cloudera, Inc. All rights reserved.
Modern data warehouse landscape
Modern Data Platform
15© Cloudera, Inc. All rights reserved.
Powered by the best-of-breed technologies
Fastest ETL/ELT at Scale
for Data Engineers
• Flexible and scalable to handle any and all
• Fast data processing with distributed, in-
• Processed data immediately available with
shared storage and metadata
• Cloud-native for contention-free resourcing
Self-Service BI & Reporting
for Analysts & SQL Developers
• Query data directly without rigid data
• Interactive multi-user performance for
• Elastic scalability for more users/data on-
premises and cloud environments
• Cloud-native for insights over shared data
16© Cloudera, Inc. All rights reserved.
Cloudera’s goal: customer success with open source
By innovating in open source
Some vendors consume the open source community’s activity; others help drive it.
Cloudera leads in influencing the Hadoop platform's evolution by creating, contributing,
and supporting new capabilities that meet customer requirements for security, scale, and
By curating open standards
Cloudera has a long and proven track record of identifying, curating, and supporting the
open standards (including Apache HBase, Apache Spark, and Apache Kafka) that
provide the mainstream, long-term architecture upon which new customer use cases
By meeting the highest enterprise requirements
To ensure the best customer experience, Cloudera invests significant resources in multi-
dimensional testing on real workloads before releases, as well as in supportability of the
entire platform via extensive involvement in the open source community.
17© Cloudera, Inc. All rights reserved.
Live Demo CDSW – Spark Data Pipelines
heute 10:20-10:30 / Cloudera Stand @ TDWI
Live Demo Altus “Job First” Big Data Integration
heute 13:10-13:20 / Cloudera Stand @ TDWI