Federal organizations increasingly are focused on creating environments that enable more data-driven decisions. Yet ensuring that all data is considered and is current, complete, and accurate is a tall order for most. To make data analytics meaningful to support real-world transformation, agency staff need business tools that provide user-friendly dashboards, on-demand reporting, and methods to manage efficiently the rise of voluminous and varied data sets and types commonly associated with big data. In most cases, existing systems are insufficient to support these requirements. Enter the enterprise data hub (EDH), a software architecture specifically designed to be a unified platform that can economically store unlimited data and enable diverse access to it at scale. Plan to attend this discussion to understand the key considerations to making an EDH the architectural center of your agency’s modern data strategy.

1. 1© Cloudera, Inc. All rights reserved.
Standing Up an
Effective Data Hub
Al Domingo, Director, Sales Engineering

2. 2© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
Our relationship with data
is changing.

3. 3© Cloudera, Inc. All rights reserved.
Data Transforming Government
CITIZEN & CHANNEL DATA-DRIVEN PRODUCTS SECURITY, RISK & COMPLIANCE

4. 4© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
60%
50%
of big data projects will fail
to go beyond the pilot phase.
or fewer organizations will have made the
cultural or business model adjustments
to benefit from big data.
Gartner “Predicts 2015: Big Data Challenges Move From Technology to the Organization” – November 2014
50%of business ethics violations will be from
improper use of big data analytics.

5. 5© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
Get the right architecture.
Assemble the right team.
Adopt an agile approach.

6. 6© Cloudera, Inc. All rights reserved.
What is Different?
1. Economically feasible to store more data
2. Powered to predictably process large data sets
3. Ability to build your data asset at linear scale
4. Collect data in native format – enables agility
5. Build history of activity by collecting data prior to its use
6. You can have near real-time access to data, plus a view of history
7. Create community data by sharing across your business
8. Out-of-the-box thinking and fail-fast increases innovation
Extreme performance
and efficiency
Analytic agility

7. 7© Cloudera, Inc. All rights reserved.
Schema on Read is the Change Agent
When storing data in Hadoop it is not necessary to
declare its structure or association with any
particular application
©2014 Cloudera, Inc. All rights reserved.
Schema on Write
• Determine Requirements
• Design Schema
• Collect & Transform Data
• Validate Design
Schema on Read
• Explore
• Transform
• Analyze
• Iterate
Image source: “Business Process Analytics” by M. Zur Muhlen, Robert Shapiro, in Handbook of
Business Process Management 2, Springer Berlin Heidelberg, pp 137-157, 2010.

8. 8© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
Get the right architecture.
Assemble the right team.
Adopt an agile approach.

9. 9© Cloudera, Inc. All rights reserved.
Current Data Architectures
Limited data. Single access. Platform silos.
SERVE
INTEGRATE
& PROCESS
ANALYZE

10. 10© Cloudera, Inc. All rights reserved.
Enterprise Data Hub
Unlimited data. Diverse access. One platform.
OPERATIONS
DATAMANAGEMENT
UNIFIED SERVICES
PROCESS,ANALYZE, SERVE
STORE
INTEGRATE
SERVE
INTEGRATE
& PROCESS
ANALYZE

11. 11© Cloudera, Inc. All rights reserved.
OPERATIONS
DATA
MANAGEMENT
BATCH REAL-TIME
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT SECURITY
FILESYSTEM RELATIONAL NoSQL
STORE
INTEGRATE
BATCH STREAM SQL SEARCH SDK
Cloudera Enterprise
Making Hadoop Fast, Easy, and Secure
A new kind of
data platform.
• One place for unlimited data
• Unified data access
Cloudera makes it:
• Fast for business
• Easy to manage
• Secure without compromise

12. 12© Cloudera, Inc. All rights reserved.
Fast for Business
More insights for more users, at lower TCO.
In-Memory Data Processing
Apache Spark
MPP Analytic SQL, Built for BI
Impala
Native Search
Apache Solr
Updateable Analytic Storage
Kudu
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee
Optimizer
BATCH
Sqoop
REAL-TIME
Kafka, Flume
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
STORE
INTEGRATE
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Partners
BATCH
Sqoop
REAL-TIME
Kafka, Flume
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Partners
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee

13. 13© Cloudera, Inc. All rights reserved.
Easy to Manage
Hadoop is complex. We make it simple, so you can focus on results.
OPERATIONS
Cloudera Manager
Cloudera Director
STRUCTURED
Sqoop
UNSTRUCTURED
Kafka, Flume
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
STORE
INTEGRATE
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Kite
BATCH
Sqoop
REAL-TIME
Kafka, Flume
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Partners
OPERATIONS
Cloudera Manager
Cloudera Director
Powerful Cluster Operations
Cloudera Manager
Hybrid Cloud Operations
Cloudera Director across
Amazon, Google, Microsoft
Predictive, Proactive Help
Cloudera Support

14. 14© Cloudera, Inc. All rights reserved.
Secure without Compromise
Security and compliance are not “opt-in” activities.
Enterprise Encryption
Cloudera Navigator Encrypt, Key Trustee
Role-Based Access Policy
Enforcement, Data Masking
Apache Sentry, RecordService
Automated Data Management
Cloudera Navigator delivers full-stack audit,
lineage, discovery, and lifecycle management
Secure Operations
Cloudera Manager provides separation of
duties, log data redaction
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee
Optimizer
STRUCTURED
Sqoop
UNSTRUCTURED
Kafka, Flume
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
STORE
INTEGRATE
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Kite
BATCH
Sqoop
REAL-TIME
Kafka, Flume
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Partners
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee
Optimizer

15. 15© Cloudera, Inc. All rights reserved.
Partner Focused. Partner Engineered.
Data
Systems
Applications
Operational
Tools
Infrastructure
System
Integration
Enterprise Data Hub
Security and Administration
Unlimited Storage
Process Discover Model Serve

16. 16© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
Get the right architecture.
Assemble the right team.
Adopt an agile approach.

17. 17© Cloudera, Inc. All rights reserved.
Executive Sponsor
Understand difference: strategic leadership and vision
• Focused on change, and willing to take risk.
• Navigate complex organizational structures.
• Collaborate to get buy-in
• Diplomatically challenge the status quo.
• Build big data success stories from within the business.
• Advocate for the use of data in new ways.
• Support the proactive collection of data and lead the
charge to assign value to data.
PROFILE
ROLE

18. 18© Cloudera, Inc. All rights reserved.
ANALYTICS
DATA MGMT
INFRASTRUCTURE
Big Data
Management
Assemble the Data Team
Tightly aligned, Mix of experts and innovators
Traditional BI and Analytics Big Data Analytics
DATA SCIENTISTS
DATA ENGINEERS
ARCHITECTS

19. 19© Cloudera, Inc. All rights reserved.
Your Infrastructure Team
Hadoop and the Big Data technology ecosystems change rapidly
– infrastructure architecture is a critical component of your
team. Architects need to balance tactical and strategic needs.
Communication
 The software and hardware infrastructure is often physically operated
by an different group. The Architect needs close collaboration.
Education
 Continually explore new technologies, including 3rd party tools –
architects need to stay ahead of the curve. Training is essential:
admin, developer.
Leadership
 Be the infrastructure expert and advise on new projects and new
requirements from the data management team and the business.
Know when to call in the experts on Hadoop and Big Data (RSA / DSE).
Description

20. 20© Cloudera, Inc. All rights reserved.
Your Data Engineer Team
Data is only useful if users can employ it in a meaningful way.
Data engineers have to be committed to making your company’s
data the utmost strategic asset, from acquisition to advocacy.
Communication
 Document, secure, audit the data. Create simple schemas and search
indexes for each data set. Create common profiles, and continually
advocate for new data and for improved data.
Education
 Get trained and certified with Cloudera Administrator, Developer,
Data Analyst courses and become an expert with Navigator, Sentry
Leadership
 Promote and evangelize to educate on the value of Big Data, take the
lead on data governance – love the data
Description

21. 21© Cloudera, Inc. All rights reserved.
The Data Scientist
Curiosity
Math &
Statistical
Knowledge
Hacking
skills
Subject
Matter
Expertise
The hybrid data scientist
• Subject Matter Expertise lies
in the business
• Hacking skills can come from
existing IT staff or new hires
• Staff at least one true Ph.D
statistician for model
oversight across all teams
Important character trait
Data Science

22. 22© Cloudera, Inc. All rights reserved.
Often, a centralized Data Science team can partner with the
business to identify data that differentiates, explore use cases to
solve, and help to jumpstart business teams, but be mindful.
Staff For Success: Data Science-as-a-service
Agility  The team must be able to learn quickly and adapt
Skills
 Hybrid skills of computer science (hacking), domain expertise and
at least one true statistician. Cloudera Data Science training.
Teams
 Often business find the domain expertise in-house, add in MS/Ph.D.
candidates from local universities and hire that one true statistician
Experts
 This team must be the “data experts” for the entire company in order
to fulfill the vision of sharing data for maximum innovation
Description

23. 23© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
Get the right architecture.
Assemble the right team.
Adopt an agile approach.

24. 24© Cloudera, Inc. All rights reserved.
Adopt an Agile Approach
Successful projects start small, Fail often
and iterate to success
Get Data
Explore
and Analyze
Deploy
1. Get data you already have, or
create new data.
2. Explore and analyze, quickly.
3. Deploy your application.
…and repeat. Add:
More data, more users, more use cases,
more complex analytics; go real-time!

25. 25© Cloudera, Inc. All rights reserved.
Operationalize Insights
Analytics ⇾ Reports, Big Data ⇾ Actions; Create bridge between
Dev and Ops
• Existing IT support handles Infrastructure for
L1, L2, L3.
• Existing IT support passes data and insights
through L1 and L2 to a DevOps team for Level
3 support. Key contact with Cloudera support
and DSEs.
• Data engineers and scientist teams work with
DevOps for production, and DevOps works
with L2 to debug and deploy.
DATA SCIENTISTS
DATA ENGINEERS
ARCHITECTS
**Cloudera administrator, developer,
navigator, security training

26. 26© Cloudera, Inc. All rights reserved.
Determine Your Data Sharing Model
Best practice is an environment that supports collaboration.
Trusted power users have broad access and new tools. Traditional
analysts have narrow access with traditional BI tools.
Data
Ingest
Data Analytics Sandbox Data Analytics Warehouse
• Data mining & discovery
• Small set of power users
• Fairly open access controls
• Minimal governance
• Pre-defined datasets
• Larger group of analysts
• Restricted access controls
• Deep & broad governance
• Data models
• Data science
• Business
Intelligence
Central Data Asset
Enterprise
Data Hub
Description

27. 27© Cloudera, Inc. All rights reserved.
Getting Started
Operational Efficiency New Business Value
OPERATIONS
DATAMANAGEMENT
UNIFIED SERVICES
PROCESS,ANALYZE, SERVE
STORE
INTEGRATE
Store and process unlimited
data fast and cost-effectively.
Data
Engineering
Explore, analyze, and
understand all your data.
Data Discovery
and Analytics
Build data-driven products
to deliver real-time insights.
Data
Applications

28. 28© Cloudera, Inc. All rights reserved.
One Platform. Many Applications.
Data
Engineering
Data Discovery
& Analytics
Data
Applications
Drive Citizen &
Mission Insights
Improve Product &
Services Efficiency
Lower
Risks
Cloudera Enterprise: Fast, Easy, Secure
Business
Value
Technology
Use Cases

29. 29© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
Get the right architecture.
Assemble the right team.
Adopt an agile approach.

30. 30© Cloudera, Inc. All rights reserved.
Thank You