Page 1 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
HDP Advanced Security
Comprehensive Security for Enterprise Hadoop
June 24, 2014
Hortonworks. We do Hadoop.
Page 2 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Delivery of YARN has led to greater demand
for the “Data Lake” architecture
•  More Workloads
From batch to interactive & real-time
•  More Data
Multiple data sets, across deeper
sets of data
•  More Value
Hosting multiple business cases
in a single Hadoop cluster
YARN Has Accelerated Hadoop Adoption
Summer 2014
65% of clusters host
multiple workloads
Fall 2013
Largely silo’d
deployments with single
workload clusters
An ever increasing proportion of
our customers are moving down
this path…
Page 3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
A Blueprint for Enterprise Hadoop
Our leadership role: enable this architecture by delivering the core capabilities for
Enterprise Hadoop across all 5 aspects of the platform completely in open source
Load data
and manage
according
to policy
Deploy and
effectively
manage the
platform
Store and process all of your Corporate Data Assets
	
  
Access your data simultaneously in multiple ways
(batch, interactive, real-time) Provide layered
approach to
security through
Authentication,
Authorization,
Accounting, and
Data Protection
	
  
DATA	
  	
  MANAGEMENT	
  
SECURITY	
  DATA	
  	
  ACCESS	
  
GOVERNANCE	
  &	
  
INTEGRATION	
  
OPERATIONS	
  
Enable both existing and new application to
provide value to the organization
PRESENTATION	
  &	
  APPLICATION	
  
Empower existing operations and
security tools to manage Hadoop
ENTERPRISE	
  MGMT	
  &	
  SECURITY	
  
Provide deployment choice across physical, virtual, cloud
DEPLOYMENT	
  OPTIONS	
  
YARN	
  :	
  Data	
  Opera:ng	
  System	
  
Page 4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Hadoop Security Requires a Layered
Approach
Load
data and
manage
accordin
g
to policy
Deploy
and
effectivel
y
manage
the
platformStore and process all of your Corporate
Data Assets
	
  
Access your data simultaneously in
multiple ways
(batch, interactive, real-time) Provide layered
approach to
security through
Authentication,
Authorization,
Accounting, and
Data Protection
	
  
DATA	
  	
  MANAGEMENT	
  
SECURITY	
  DATA	
  	
  ACCESS	
  
GOVERNANCE	
  
&	
  
INTEGRATION	
  
OPERATIONS	
  
Enable both existing and new application
to
provide value to the organization
PRESENTATION	
  &	
  APPLICATION	
  
Empower existing
operations and
security tools to manage
Hadoop
ENTERPRISE	
  MGMT	
  &	
  SECURITY	
  
Provide deployment choice across
physical, virtual, cloud
DEPLOYMENT	
  OPTIONS	
  
YARN	
  :	
  Data	
  Opera:ng	
  System	
  
COMPREHENSIVE SECURITY
Meet all security requirements
across authentication,
authorization, audit & data
protection
CENTRALIZED
ADMINISTRATION
Provide one location for
administering security policies and
for viewing and managing audit
across the platform
CONSISTENT INTEGRATION
Integrate with other security and
identity management systems, for
compliance with IT policies
Founded in 2013, XA Secure
provides an enterprise ready,
cross-platform, security
solution built from the ground
up for Hadoop, providing
centralized capabilities around
data security, authorization,
audit and governance.
Hortonworks
has acquired XA Secure
Acquisition will accelerate delivery of
enterprise-grade centralized security
administration and enforcement across
batch, interactive, and real-time workloads
running in Hadoop
Page 5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Broader Security Needs –
Representative Use Case
Marketing
Cust Name Phone # Email
HR
Employee# Name Location
Finance
Account Date Amount#
HDP
Platform
Result Set
Cust Name Employee# Transaction#
Result Set
Account Date Amount#
Analyst team
has access to
all data except
PII
HR auditors
need access
history for all
users, specific
resources
Finance team
has access
only to finance
data
Page 6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Broader Security Needs –
Representative Use Case
HR FinanceMarketing
HDP
Platform
Data Scientist
team is
running a
MapReduce,
should not
have access
to PII files
HR audit
policy entails
auditing of
privileged
resources
HR team has
access only to
HR folder
Page 7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Security in Hadoop with HDP Advanced Security
Authorization
Restrict data
access
Audit
Know who did
what
Centralized
Administration
•  Kerberos in
native Apache
Hadoop
•  HTTP/REST
API Secured
with Apache
Knox Gateway
Authentication
Prove who I am
•  UI to manage
security
policies
•  Delegated
Administration
•  Automated
policy push
•  HDFS, Hive
and Hbase:
fine grain
access control
•  Role-based
policies
•  Component-
level
enforcement
•  Centralized
audit reporting
•  Detailed
access
auditing
•  Admin action
auditing
HDP2.1
XASecure
Page 8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
HDP Advanced Security - Features
XA Secure + HDP
Authentication
Kerberos Support ✔
Perimeter Security – For services
and rest API
✔
Authorizations
Fine grained access control HDFS, HBase and
Hive
Role base access control ✔
Column level ✔
Permission Support Create, Drop, Index,
lock, user
Page 9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
XA Secure + HDP
Auditing
Resource access auditing Extensive Auditing
Policy auditing ✔
Reporting
Configurable reporting tool ✔
Manage
User/ Group mapping ✔
Central policy distribution ✔
Global policy manager, Web UI ✔
Delegated administration ✔
HDP Advanced Security - Features
Page 10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Hortonworks Delivers Comprehensive
Hadoop Security
Acquisition accelerates
delivery of comprehensive
security for Hadoop across
batch, interactive, and real-
time workloads running in
Hadoop
Underscores Hortonworks
commitment to deliver
enterprise Hadoop
completely in open source
June
2 Demonstrate and share
more details on the
technology. Technology
Available & Supported
June
24
HDP Security
Webinar
Public Demonstration &
HDP Tech Preview
Available
2H
2014
ASF Incubation
Incubate technology as
an open Apache Project
Page 11 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Demo
Hortonworks. We do Hadoop.
Page 12 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Download HDP Advanced Security
Hortonworks.com/labs/security/
Get It
hortonworks.com/hdpaddons/
Try It with Hortonworks Sandbox
http://hortonworks.com/hadoop-tutorial/
securing-data-lake-auditing-user-access-
using-hdp-security/
Page 13 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Thank you!

HDP Advanced Security: Comprehensive Security for Enterprise Hadoop

  • 1.
    Page 1 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved HDP Advanced Security Comprehensive Security for Enterprise Hadoop June 24, 2014 Hortonworks. We do Hadoop.
  • 2.
    Page 2 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Delivery of YARN has led to greater demand for the “Data Lake” architecture •  More Workloads From batch to interactive & real-time •  More Data Multiple data sets, across deeper sets of data •  More Value Hosting multiple business cases in a single Hadoop cluster YARN Has Accelerated Hadoop Adoption Summer 2014 65% of clusters host multiple workloads Fall 2013 Largely silo’d deployments with single workload clusters An ever increasing proportion of our customers are moving down this path…
  • 3.
    Page 3 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved A Blueprint for Enterprise Hadoop Our leadership role: enable this architecture by delivering the core capabilities for Enterprise Hadoop across all 5 aspects of the platform completely in open source Load data and manage according to policy Deploy and effectively manage the platform Store and process all of your Corporate Data Assets   Access your data simultaneously in multiple ways (batch, interactive, real-time) Provide layered approach to security through Authentication, Authorization, Accounting, and Data Protection   DATA    MANAGEMENT   SECURITY  DATA    ACCESS   GOVERNANCE  &   INTEGRATION   OPERATIONS   Enable both existing and new application to provide value to the organization PRESENTATION  &  APPLICATION   Empower existing operations and security tools to manage Hadoop ENTERPRISE  MGMT  &  SECURITY   Provide deployment choice across physical, virtual, cloud DEPLOYMENT  OPTIONS   YARN  :  Data  Opera:ng  System  
  • 4.
    Page 4 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Hadoop Security Requires a Layered Approach Load data and manage accordin g to policy Deploy and effectivel y manage the platformStore and process all of your Corporate Data Assets   Access your data simultaneously in multiple ways (batch, interactive, real-time) Provide layered approach to security through Authentication, Authorization, Accounting, and Data Protection   DATA    MANAGEMENT   SECURITY  DATA    ACCESS   GOVERNANCE   &   INTEGRATION   OPERATIONS   Enable both existing and new application to provide value to the organization PRESENTATION  &  APPLICATION   Empower existing operations and security tools to manage Hadoop ENTERPRISE  MGMT  &  SECURITY   Provide deployment choice across physical, virtual, cloud DEPLOYMENT  OPTIONS   YARN  :  Data  Opera:ng  System   COMPREHENSIVE SECURITY Meet all security requirements across authentication, authorization, audit & data protection CENTRALIZED ADMINISTRATION Provide one location for administering security policies and for viewing and managing audit across the platform CONSISTENT INTEGRATION Integrate with other security and identity management systems, for compliance with IT policies Founded in 2013, XA Secure provides an enterprise ready, cross-platform, security solution built from the ground up for Hadoop, providing centralized capabilities around data security, authorization, audit and governance. Hortonworks has acquired XA Secure Acquisition will accelerate delivery of enterprise-grade centralized security administration and enforcement across batch, interactive, and real-time workloads running in Hadoop
  • 5.
    Page 5 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Broader Security Needs – Representative Use Case Marketing Cust Name Phone # Email HR Employee# Name Location Finance Account Date Amount# HDP Platform Result Set Cust Name Employee# Transaction# Result Set Account Date Amount# Analyst team has access to all data except PII HR auditors need access history for all users, specific resources Finance team has access only to finance data
  • 6.
    Page 6 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Broader Security Needs – Representative Use Case HR FinanceMarketing HDP Platform Data Scientist team is running a MapReduce, should not have access to PII files HR audit policy entails auditing of privileged resources HR team has access only to HR folder
  • 7.
    Page 7 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Security in Hadoop with HDP Advanced Security Authorization Restrict data access Audit Know who did what Centralized Administration •  Kerberos in native Apache Hadoop •  HTTP/REST API Secured with Apache Knox Gateway Authentication Prove who I am •  UI to manage security policies •  Delegated Administration •  Automated policy push •  HDFS, Hive and Hbase: fine grain access control •  Role-based policies •  Component- level enforcement •  Centralized audit reporting •  Detailed access auditing •  Admin action auditing HDP2.1 XASecure
  • 8.
    Page 8 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved HDP Advanced Security - Features XA Secure + HDP Authentication Kerberos Support ✔ Perimeter Security – For services and rest API ✔ Authorizations Fine grained access control HDFS, HBase and Hive Role base access control ✔ Column level ✔ Permission Support Create, Drop, Index, lock, user
  • 9.
    Page 9 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved XA Secure + HDP Auditing Resource access auditing Extensive Auditing Policy auditing ✔ Reporting Configurable reporting tool ✔ Manage User/ Group mapping ✔ Central policy distribution ✔ Global policy manager, Web UI ✔ Delegated administration ✔ HDP Advanced Security - Features
  • 10.
    Page 10 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Hortonworks Delivers Comprehensive Hadoop Security Acquisition accelerates delivery of comprehensive security for Hadoop across batch, interactive, and real- time workloads running in Hadoop Underscores Hortonworks commitment to deliver enterprise Hadoop completely in open source June 2 Demonstrate and share more details on the technology. Technology Available & Supported June 24 HDP Security Webinar Public Demonstration & HDP Tech Preview Available 2H 2014 ASF Incubation Incubate technology as an open Apache Project
  • 11.
    Page 11 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Demo Hortonworks. We do Hadoop.
  • 12.
    Page 12 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Download HDP Advanced Security Hortonworks.com/labs/security/ Get It hortonworks.com/hdpaddons/ Try It with Hortonworks Sandbox http://hortonworks.com/hadoop-tutorial/ securing-data-lake-auditing-user-access- using-hdp-security/
  • 13.
    Page 13 ©Hortonworks Inc. 2011 – 2014. All Rights Reserved Thank you!