Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Page1 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Apache Hadoop Security: Ranger
Sep 16, 2015
Madhan Neethiraj
Page2 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Agenda
Control
access into
system
Flexibility
in defining
polici...
Page3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Security in Hadoop
Authentication
Authenticate users and systems...
Page4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Authorization and Auditing with Ranger
HBase
Ranger Administrati...
Page5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Central Security Administration
Apache Ranger
• Delivers a ‘sing...
Page6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ranger Authorization
Ranger Plugins authorize access to resource...
Page7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ranger Auditing
• Ranger plugins generate detailed audit logs fo...
Page8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ranger Policy - Hive
Allow Marketing group users ‘select’ access...
Page9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ranger Policy - HDFS
Allow Marketing group users to access /demo...
Page10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ranger Policy - Kafka
Allow Marketing group users to access Mar...
Page11 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensible Architecture
Page12 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Ranger Stacks
• Customers and partners can easil...
Page13 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Ranger Stacks - example
Page14 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Dynamic Policy Conditions
• Provides ability to ...
Page15 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Dynamic Policy Conditions - sample
• Register th...
Page16 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Context Enrichers
• Provides ability to add cont...
Page17 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Context Enrichers - sample
• Register the contex...
Page18 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Extensibility: Context Enricher + Condition - sample
• Implemen...
Page19 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
In Development: allow/deny/exceptions in policies
• Ability to ...
Page20 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Features under Development..
Page21 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
In Development: allow/deny/exceptions in policies
allow access ...
Page22 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
In Development: tag-based policies
• Ability authorize access b...
Page23 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
In Development: tag-based policies
Policy to authorize access t...
Page24 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Apache Ranger: how to contribute?
• Ranger Home Page - http://r...
Page25 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Q&A…
Discussion
Upcoming SlideShare
Loading in …5
×

Apache Hadoop Security - Ranger

2,394 views

Published on

Slides on Apache Hadoop Security - Ranger

Published in: Technology
  • ⇒ www.HelpWriting.net ⇐ This service will write as best as they can. So you do not need to waste the time on rewritings.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I like this service ⇒ www.WritePaper.info ⇐ from Academic Writers. I don't have enough time write it by myself.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • i used Auto Profit Replicator System for 1 week, its give me profits but only $79 :(. I think Mikes Auto Trader Software is better than it. If you want to Check Mikes Auto Trader Software than you can get it free from here>> tr.im/mikeautotrader
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Apache Hadoop Security - Ranger

  1. 1. Page1 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Apache Hadoop Security: Ranger Sep 16, 2015 Madhan Neethiraj
  2. 2. Page2 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Agenda Control access into system Flexibility in defining policies • Authorization & Auditing with Ranger • Centralized security administration for HDFS, Hive, HBase, Knox, Strom, YARN, Kafka, Solr, .. • Audit logs to Solr, HDFS, RDBMS, Log4j, .. • Extensible Architecture – custom conditions, context enrichers, easier addition of new components
  3. 3. Page3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Security in Hadoop Authentication Authenticate users and systems Apache Knox, Native Kerberos Authorization Provision access to data Apache Ranger Audit Maintain a record of data access Apache Ranger, Hadoop native audit Data Protection Protect data at rest and in motion HDFS encryption + Ranger KMS, Vendor solutions Administration Central management & consistent security Apache Ranger
  4. 4. Page4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Authorization and Auditing with Ranger HBase Ranger Administration Portal HDFS Hive Server2 Ranger Policy Store Ranger Audit Store Ranger Plugin Hadoop Components Enterpris e Users Log4j Knox Storm RDBMS YARN Kafka Solr HDFS Solr Ranger Plugin Ranger Plugin Ranger Plugin Ranger Plugin Ranger Plugin Ranger Plugin Ranger Plugin
  5. 5. Page5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Central Security Administration Apache Ranger • Delivers a ‘single pane of glass’ for the security administrator • Centralizes administration of security policy • Ensures consistent coverage across the entire Hadoop stack
  6. 6. Page6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Ranger Authorization Ranger Plugins authorize access to resources in following Hadoop components: Component Resources Access Types HDFS Files/Directories Read, Write, Execute Hive Databases, Tables, Columns Create, Alter, Drop, Select, Update, All HBase Tables, Column-Families, Columns Read, Write, Create, Admin Knox Topologies, Services Allow Storm Topologies Topology: submit/activate/deactivate/reblance/kill/get/get- info/get-user/get-conf, File: upload/download, Get Nimbus Conf YARN Queues Submit-application, Admin-queue Kafka Topics Publish, Consume, Configure, Describe, Admin Solr Collections Query, Update, Others, Admin
  7. 7. Page7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Ranger Auditing • Ranger plugins generate detailed audit logs for accesses to protected resources. Audit logs include details like: user, resource, type of access, time of access, client IP address, access-result, ID of the policy that allowed/denied the access • Audit logs to one or more destinations – Solr, HDFS, RDBMS, Log4j, ... • Interactive view of audit logs using Ranger Admin
  8. 8. Page8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Ranger Policy - Hive Allow Marketing group users ‘select’ access on few columns in customer_details table
  9. 9. Page9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Ranger Policy - HDFS Allow Marketing group users to access /demo/data/Customer* directories and files
  10. 10. Page10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Ranger Policy - Kafka Allow Marketing group users to access Marketing topic
  11. 11. Page11 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensible Architecture
  12. 12. Page12 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Ranger Stacks • Customers and partners can easily add Ranger authorization and auditing support for new components • Describe component details (like resource structure, access-types) in JSON and register with Ranger • Implement component authorizer to authorize resource accesses using Ranger policy engine • Ranger Admin provides UI for policy administration, based on component details in registered JSON
  13. 13. Page13 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Ranger Stacks - example
  14. 14. Page14 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Dynamic Policy Conditions • Provides ability to evaluate custom conditions to drive authorization decisions • Custom conditions can evaluate various data available in the request – like user, groups, resource, IP-address, context, etc. • Register custom conditions via component description JSON • Ranger Admin provides UI to specify condition values to be satisfied Allow accesses from 10.0.2.* IP addresses only!
  15. 15. Page15 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Dynamic Policy Conditions - sample • Register the custom condition in the component description JSON: • Implement the custom condition and make it available to Ranger plugin: • Ranger Policy Engine will call the custom condition while evaluating policies
  16. 16. Page16 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Context Enrichers • Provides ability to add context data to access requests • Context data added can be used by condition evaluators to drive authorization decisions • An example: from the client IP address in the request, a context enricher adds location data (like COUNTRY, STATE, CITY, AREA-CODE) to the request context. A custom condition can then restrict access depending upon the location data in the context. • Context enrichers should be specified in component description JSON
  17. 17. Page17 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Context Enrichers - sample • Register the context enrichers in the component description JSON: • Implement the context enricher and make it available to Ranger plugin: • Ranger Policy Engine will call all registered context enrichers before evaluating policies
  18. 18. Page18 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Extensibility: Context Enricher + Condition - sample • Implement a custom condition that verifies that the access is from specified countries only: • Register the custom condition in the component description JSON • On receiving authorization request, Ranger Policy Engine calls LocationDataProviderEnricher enricher, which adds location data to the request. • When evaluating policies, Ranger Policy Engine calls LocationCountryCondition, which allows accesses only from the countries specified in the policy
  19. 19. Page19 © Hortonworks Inc. 2011 – 2014. All Rights Reserved In Development: allow/deny/exceptions in policies • Ability to explicitly deny access to resources • Ability to allow/deny access to a wider group, like employees/public, but specify exceptions to a subset, like part-time employees/vendors/ip- addresses, etc. • Policy evaluation order: • All deny-policies for the resource are evaluated first • If the request matches a deny-policy, and not its deny-exceptions, access will be denied • If the request is not denied by deny-policies, allow-policies will be evaluated • If the request matches an allow-policy, and not its allow-exceptions, access will be allowed • Development in tag-policy branch of Apache Ranger
  20. 20. Page20 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Features under Development..
  21. 21. Page21 © Hortonworks Inc. 2011 – 2014. All Rights Reserved In Development: allow/deny/exceptions in policies allow access to finance group and falcon user deny access from outside of Switzerland for everyone, except falcon user Policy to: - deny access from outside Switzerland to everyone, except falcon user - allow falcon user to access from anywhere - allow finance group users to access from Switzerland only
  22. 22. Page22 © Hortonworks Inc. 2011 – 2014. All Rights Reserved In Development: tag-based policies • Ability authorize access based on tags associated with resources • A single tag-based policy, like for PII tag, to authorize access to resources across components – like HDFS, Hive, HBase, .. • Available to all components that use Ranger authorization • Similar policy structure as existing resource-based policies • API to integrate with tag providers – like Apache Atlas • Development in tag-policy branch of Apache Ranger
  23. 23. Page23 © Hortonworks Inc. 2011 – 2014. All Rights Reserved In Development: tag-based policies Policy to authorize access to resources tagged as PII, in HDFS/Hive/HBase/Kafka/Solr, only to audit users
  24. 24. Page24 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Apache Ranger: how to contribute? • Ranger Home Page - http://ranger.incubator.apache.org • Ranger Wiki - https://cwiki.apache.org/confluence/display/RANGER • Ranger JIRAs - https://issues.apache.org/jira/browse/RANGER • Project Mailing Lists • Users: user@ranger.incubator.apache.org • Developers: dev@ranger.incubator.apache.org • Commits: commits@ranger.incubator.apache.org
  25. 25. Page25 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Q&A… Discussion

×