Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Enterprise Ready Security & Governance
with Hortonworks Data Platfor...
2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Protecting the Elephant in the Castle…..
Kerberos,
Wire Encryption
H...
3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger
• Central audit location for all
access requests
• Sup...
4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Ranger Architecture
HDFS
Ranger Administration Portal
HBase
Hive Ser...
5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Enterprise Data Governance: Apache Atlas
Data Management
along the e...
6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
High Level Architecture: 4 Key points
Type System
Repository
Search ...
7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Atlas Component Integration
• Cross- component dataset lineag...
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Next Generation Security & Governance for Hadoop
9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Demo Scenario
 HortoniaBank – mid-size financial services company (...
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Demo Data
 Customer data in hortoniabank DB
• 2 Customer Tables: 5...
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Ranger Policies Setup for Demo
 Only US employees can see data in ...
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Personas Setup for Demo
User Group Access Privileges
joe-analyst us...
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Data Column Data
Column
Description
Masking
Type
Sample Output Rang...
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Tag Based Policy for Leased data
Group Access Privileges
public No ...
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
HDP Security Benefits Comprehensive Security
through a platform app...
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the Enterprise
Next
Upcoming SlideShare
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the Enterprise
Next
Download to read offline and view in fullscreen.

Share

Security and Governance on Hadoop with Apache Atlas and Apache Ranger by Srikanth Venkat

Download to read offline

Security and governance on hadoop with apache atlas and apache ranger by srikanth venkat

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Security and Governance on Hadoop with Apache Atlas and Apache Ranger by Srikanth Venkat

  1. 1. 1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Enterprise Ready Security & Governance with Hortonworks Data Platform Srikanth Venkat Senior Director, Product Management
  2. 2. 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Protecting the Elephant in the Castle….. Kerberos, Wire Encryption HDFS Encryption Apache Ranger Network Segmentation, Firewalls LDAP/AD Apache Knox
  3. 3. 3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Ranger • Central audit location for all access requests • Support multiple destination sources (HDFS, Solr, etc.) • Real-time visual query interface AuditingAuthorization • Store and manage encryption keys • Support HDFS Transparent Data Encryption • Integration with HSM • Safenet LUNA Ranger KMS • Centralized platform to define, administer and manage security policies consistently across Hadoop components • HDFS, Hive, HBase, YARN, Kafka, Solr, Storm, Knox, NiFi • Extensible Architecture • Custom policy conditions, user context enrichers • Easy to add new component types for authorization
  4. 4. 4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Ranger Architecture HDFS Ranger Administration Portal HBase Hive Server2 Ranger Audit Server Ranger Plugin HadoopComponentsEnterprise Users Ranger Plugin Ranger Plugin Legacy Tools and Data Governance HDFS Knox NifI Ranger Plugin Ranger Plugin SolrRanger Plugin Ranger Policy Server Integration API KafkaRanger Plugin YARNRanger Plugin Ranger PluginStorm Ranger Plugin Atlas Solr
  5. 5. 5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Enterprise Data Governance: Apache Atlas Data Management along the entire data lifecycle with integrated provenance and lineage capability • Cross component lineage Modeling with Metadata enables comprehensive business metadata vocabulary with enhanced tagging and attribute capabilities • Common Business Language • Hierarchically organized – No dupes ! Interoperable Solutions across the Hadoop ecosystem, through a common metadata store • Combine and Exchange Metadata STRUCTURED TRADITIONAL RDBMS METADATA MPP APPLIANCES Kafka Storm Sqoop Hive ATLAS METADATA Falcon RANGER Custom Partners
  6. 6. 6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved High Level Architecture: 4 Key points Type System Repository Search DSL Bridge Hive Storm Falcon Custom REST API Graph DB Search Kafka Sqoop Connectors MessagingFramework 3 REST API Modern, flexible access to Atlas services, HDP components, UI & external tools 1 Data Lineage Only product that captures lineage across Hadoop components at platform level. 4 Exchange Leverage existing metadata / models by importing it from current tools. Export metadata to downstream systems 2 Agile Data Modeling: Type system allows custom metadata structures in a hierarchy taxonomy
  7. 7. 7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Atlas Component Integration • Cross- component dataset lineage. Centralized location for all metadata inside HDP • Single Interface point for Metadata Exchange with platforms outside of HDP Apache Atlas Hive Ranger Falcon Sqoop Storm Kafka Spark NiFi HBase HDP 2.3 HDP 2.5 Beyond HDP 2.5
  8. 8. 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Next Generation Security & Governance for Hadoop
  9. 9. 9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Demo Scenario  HortoniaBank – mid-size financial services company (bank + health insurance services) expanding from US to international markets  Employees in EU and US  Multiple business units need access to customer data: Analysts, Compliance Admins, HR  Customer data is co-mingled as well as isolated  Leases data from external data brokers  Needs to have rational security policies to provide the right level of access control to customer data across geographies, business functions, and to comply with external regulations (PII, HIPAA, EU Privacy etc.) all user passwords: hadoop
  10. 10. 10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Demo Data  Customer data in hortoniabank DB • 2 Customer Tables: 50K customer records each with 38 fields (PII, PHI, PCI & non- sensitive data) –us_customers: USA person data only –ww_customers: multi-language, multi-country, localized person data across the world • 1 Reference table: eu_countries (reference table for looking up EU country codes to country mappings – with BRExit etc.)  Finance DB: 1 data set leased from a data broker – tax_2015: Data lease expired already (on Dec 31st 2015) all user passwords: hadoop
  11. 11. 11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Ranger Policies Setup for Demo  Only US employees can see data in us_customers table and only from locations within the US (access_us_customers)  Only US employees can see data rows of US persons in ww_customers table (filter_ww_customers_table + access_ww_customers)  Only EU employees can see rows with EU person data in ww_customers table (filter_ww_customers_table + access_ww_customers)  US HR team members can see all original unmasked data (PCI, PII,….)  Analysts can view masked versions of sensitive data from WW customers table but are prohibited from viewing PII data in US tables (All masking policies under Masking Tab of Resource based policies)  No combination of zip code, MRN, and bloodgroup data are permitted to be joined in any query (prohibition policy)
  12. 12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Personas Setup for Demo User Group Access Privileges joe-analyst us_employees, analyst US Data Only, non-sensitive data only, rest masked or forbidden depending on sensitivity kate-hr us_employees, hr US Data Only, All sensitive data (PCI, PII, PHI) ivana-eu-hr eu_employees, hr EU Data Only, All sensitive data compliance-admin compliance, us_employees Compliance with licensing, can only see leased data sets
  13. 13. 13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Data Column Data Column Description Masking Type Sample Output Ranger Masking Policy password Password Hash 237672b21819462ff39fcea7d990c3e5 mask_password_hash nationalid National ID Show Last 4 xx-xx-9324 mask_nationalid_last4 ccnumber Credit Card Number Show First 4 4532xxxxxxxxxxxx mask_ccnumber_first4 streetaddress Street Address Redact nnn Xxxxxx Xxxxx mask_streetaddress_redact MRN MRN Nullify null mask_mrn_nullify age Age CUSTOM (Adds a random number below 20 to actual age) mask_age_custom birthday Date of Brith CUSTOM 01-01-1987 (Keep year of birth and make date & month 01-01) mask_dob_custom Data Masking Policies setup for us_customers data for analyst group
  14. 14. 14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Tag Based Policy for Leased data Group Access Privileges public No Access after data lease expiration date (denied) compliance Compliance team allowed to access data after expiration date  Tagging Leased Data set in Atlas  tax_2015 table tagged with EXPIRES_ON with expiry_date:2015-12-31  Tag Based Policy in Ranger for leased dataset: (Policy name: tag_EXPIRES_ON)
  15. 15. 15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved HDP Security Benefits Comprehensive Security through a platform approach. Providing Administrators with complete visibility into the security administration process Data Protection Encryption of data at rest and in motion, Dynamic Masking & Row Filtering Centralized Administration of security policies and user authentication. Consistently define, administer and manage security policies. Define a policy once and apply it to all the applicable components across the stack Fine-Grain Authorization for data access control for Database, Table, Column, LDAP Groups & Specific Users. Dynamic tag based policies Integrated with Data Governance via Apache Atlas Y A R N D A T A O P E R A T I N G S Y S T E M OPERATIONS SECURITY GOVERNANCE STORAGE STORAGE Machine Learning Batch StreamingInteractive Search SECURITY
  • MiftahurrizqiAdhiNug

    Dec. 23, 2020
  • Tala139

    Apr. 5, 2017
  • tazimehdi

    Dec. 17, 2016
  • AlexKosykh1

    Nov. 14, 2016
  • bunkertor

    Nov. 4, 2016
  • nahuelt88

    Oct. 18, 2016

Security and governance on hadoop with apache atlas and apache ranger by srikanth venkat

Views

Total views

6,612

On Slideshare

0

From embeds

0

Number of embeds

7

Actions

Downloads

161

Shares

0

Comments

0

Likes

6

×