Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Page1 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Apache Ranger
Rommel Garcia
Page2 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Who Am I
• Solutions Engineer @hortonworks
• Security SME Lead @...
Page3 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
5 Pillars of Security
• Authentication
• Authorization
• Audit
•...
Page4 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Hadoop Security Tools
• AD/LDAP (authentication)
• Apache Knox (...
Page5 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Data
Sources
Page6 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Apache Ranger
• Provides centralized policy definition for autho...
Page7 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Agent AgentAgent AgentAgent Agent
Apache Ranger authZ Architectu...
Page8 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Sample Simplified Workflow - HDFS
Policy
Manager
Agent
Admin set...
Page9 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
authZ Best Practice – POSIX + Ranger
• HDFS -> POSIX -> owned by...
Page10 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
authZ Best Practice - Ranger
10
000
(posix permissions on all H...
Page11 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Ranger UserSync Best Practice
11
• Ensure LDAPS is used to inte...
Page12 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Ranger Audit Locations
12
• HDFS
– Long term storage that can b...
Page13 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Apache Ranger – ACLs & Audit Demo
Environment
• CentOS 6.6
• 2 ...
Page14 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Q&A
Page15 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
1
°
°
°
°
° °
° °
° °
° °
° N°
Ranger KMS + HDFS TDE
DATA ACCES...
Page16 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Apache Ranger – KMS + TDE Demo
Exercise
• Create an encryption ...
Page17 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Thank you!
Rommel Garcia
@rommelgarcia
/in/rommelgarcia
Upcoming SlideShare
Loading in …5
×

Apache Ranger

4,195 views

Published on

Data in Hadoop is getting bigger every day, consumers of the data are growing, organizations are now looking at making their Hadoop cluster compliant to federal regulations and commercial demands. Apache Ranger simplifies the management of security policies across all components in Hadoop. Ranger provides granular access controls to data.

The deck describes what security tools are available in Hadoop and their purpose then it moves on to discuss in detail Apache Ranger.

Published in: Software
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❤❤❤ http://bit.ly/2F90ZZC ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating direct: ❤❤❤ http://bit.ly/2F90ZZC ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Mikes Auto Trader is a great system to make money easy. You can earn easily $300-$500 in a day. Mikes Auto Trader provide high volatility means High Return Of Investment. Click Here to go: tr.im/mikeautotrader Its provide 80% winning chance of making a profitable trade is very good. I have never come across anything like this before.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Apache Ranger

  1. 1. Page1 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Apache Ranger Rommel Garcia
  2. 2. Page2 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Who Am I • Solutions Engineer @hortonworks • Security SME Lead @hortonworks • Author “Virtualizing Hadoop: How to Install, Deploy, and Optimize Hadoop in A Virtualized Architecture”
  3. 3. Page3 © Hortonworks Inc. 2011 – 2015. All Rights Reserved 5 Pillars of Security • Authentication • Authorization • Audit • Encryption • Centralized Administration
  4. 4. Page4 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Hadoop Security Tools • AD/LDAP (authentication) • Apache Knox (authentication) • Kerberos (authentication) • Apache Ranger (authorization, audit, kms) • HDFS TDE (data encryption) • Wire Encryption (data protection)
  5. 5. Page5 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Data Sources
  6. 6. Page6 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Apache Ranger • Provides centralized policy definition for authorizing access to resources • Supported components as of v0.5 • HDFS • HBase • Hive • YARN • Knox • Storm • Solr • Kafka
  7. 7. Page7 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Agent AgentAgent AgentAgent Agent Apache Ranger authZ Architecture HBase Hive YARN Knox Storm Solr Kafka Agent HDFS Agent Audit Server Policy Server Administration Portal REST APIs DB SOLR HDFS KMS LDAP/AD user/group syncLog4j
  8. 8. Page8 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Sample Simplified Workflow - HDFS Policy Manager Agent Admin sets policies for HDFS files/folder Data scientist runs a map reduce job User Application Users access HDFS data through application Name Node IT users access HDFS through CLI Namenode uses Agent for Authorization Audit Database Audit logs pushed to DB Namenode provides resource access to user/client 1 2 2 2 3 4 5
  9. 9. Page9 © Hortonworks Inc. 2011 – 2015. All Rights Reserved authZ Best Practice – POSIX + Ranger • HDFS -> POSIX -> owned by hdfs -> Ranger ACLs • Hive -> POSIX -> owned by hive -> Ranger ACLs • HBase -> POSIX -> owned by hbase -> Ranger ACLs • Solr -> native -> owned by solr -> Ranger ACLs • Kafka -> owned by kafka -> Ranger ACLs
  10. 10. Page10 © Hortonworks Inc. 2011 – 2015. All Rights Reserved authZ Best Practice - Ranger 10 000 (posix permissions on all HDFS files)
  11. 11. Page11 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Ranger UserSync Best Practice 11 • Ensure LDAPS is used to integrate with Ranger • Create OU ONLY for Hadoop users for performance • Only run usersync when necessary – How much users are being added and how often – How much users are changing roles – Too much syncing can degrade LDAP performance • Do not sync anonymously
  12. 12. Page12 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Ranger Audit Locations 12 • HDFS – Long term storage that can be used to understand user event trends and predict anomaly • RDBMS – When SQL is preferred by auditors – MySQL, Oracle, Postgres, SQL Server • Solr – Nice quick reporting metrics to understand user event trends • Log4j Appenders
  13. 13. Page13 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Apache Ranger – ACLs & Audit Demo Environment • CentOS 6.6 • 2 vms • FreeIPA 2.0 • HDP 2.3 • Apache Ranger v0.5 • Kerberized 2 node cluster
  14. 14. Page14 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Q&A
  15. 15. Page15 © Hortonworks Inc. 2011 – 2015. All Rights Reserved 1 ° ° ° ° ° ° ° ° ° ° ° ° ° N° Ranger KMS + HDFS TDE DATA ACCESS DATA MANAGEMENT 1 ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° SECURITY YARN HDFS Client ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° °HDFS (Hadoop Distributed File System) Encryption Zone (attributes - EZKey ID, version) HDFS-6134 Encrypted File (attributes - EDEK, IV) Name Node KeyProvider API KeyProvider API Key Management System (KMS) Hadoop-10433 KeyProvider API – Hadoop-10141 EDEK DEK Crypto Stream (r/w with DEK) DEKs EZKs Acronym Description EZ Encryption Zone (an HDFS directory) EZK Encryption Zone Key; master key associated with all files in an EZ DEK Data Encryption Key, unique key associated with each file. EZ Key used to generate DEK EDEK Encrypted DEK, Name Node only has access to encrypted DEK. IV Initialization Vector EDEK EDEK
  16. 16. Page16 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Apache Ranger – KMS + TDE Demo Exercise • Create an encryption zone • Create key for encryption zone • Create file • Load to hdfs, encrypted zone • List encrypted file • Print encrypted file
  17. 17. Page17 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Thank you! Rommel Garcia @rommelgarcia /in/rommelgarcia

×