The document discusses the security landscape in Hadoop, focusing on the role and key features of Apache Ranger, which provides a centralized security framework for authentication, authorization, and auditing. It highlights the management and creation of policies for various components such as HDFS and Hive, as well as the use of audit logs for tracking access. Additionally, best practices for implementing Ranger with Hadoop's various components and tag-based policies are outlined.

1. Securing Hadoop using
Ranger
Raj Nadipalli
Director Professional Services, Zaloni
rnadipalli@zaloni.com
09.22.2016

2. Agenda
Ø Security Landscape in Hadoop
Ø Role of Ranger
Ø Ranger Key Features
Ø Demo
Ø Q&A

3. Overview

4. Security Landscape in Hadoop (open source)
Authentication
Who am I?
AD/LDAP
Kerberos
Apache Knox
Authorization
What can I do?
Apache Ranger
Apache Sentry
Audit
What happened?
Apache Ranger
Data Protection
SSL
KMS

5. Ranger in a slide
5
Ø Centralized security framework, authen*ca*on, audi*ng, data encryp*on and security
Ø Fine-grained access control over Hadoop
Ø Components Supported: HDFS, Hive, Hbase, Storm, YARN, Knox, KaCa, Solr
Ø Manage/Create policies using browser
Ø Manage Audit tracking and policy analy*cs in HDFS, RDMS or SOLR
Ø Supports governance with Tag based policies
Ø REST API’s for policy management automate, integrate and extend

6. Key Components of Ranger
http://www.slideshare.net/RommelGarcia2/apache-ranger?qid=1150145e-a144-4603-9165-
a09b2ae5ece0&v=&b=&from_search=4

7. Securing HDFS

8. Ranger in Action - HDFS
http://www.slideshare.net/RommelGarcia2/apache-ranger?qid=1150145e-a144-4603-9165-
a09b2ae5ece0&v=&b=&from_search=4

9. Ranger administration portal
9

10. List HDFS policies
10
Under HDFS policies we can view all the HDFS policies created and which user(s) / group(s) has access to which
policies
Actions
delete / edit
Policy Name
Groups/users
assigned to
policies

11. Create HDFS policy
11
Under HDFS policy we can edit/create HDFS policies, this page shows how to create a policy at user level and
provide appropriate permissions.

12. Access error in Audit
12
Under Audit tab admin can view which user tried to access which directory, here user mukesh got access denied
as it did not had the permission to access /testRanger directory
Access Denied
to user mukesh

13. List HDFS policies for group
13
Under HDFS policies we can view all the HDFS policies created and which user(s) / group(s) has
access to which policies

14. Create HDFS policy for group
14
Under HDFS policy we can edit/create HDFS policies, this page shows how to create a policy at group level and
provide appropriate permissions.
Access given
to a group

15. Securing Hive

16. List policies of Hive
16
Under Hive policies we can view all the Hive policies created and which user(s) / group(s) has access to which
policies
Hive policy for database User assigned to a policy

17. Create policy for Hive
17
Under Hive policy we can edit/create Hive policies, this page shows how to create a policy at user level and
provide appropriate permissions.

18. Access error in Audit
18
Under Audit tab admin can view which user tried to access which table/database, here user mukesh got access denied
as it did not had the permission to create table under testranger database.

19. Securing HBase

20. Create HBase policy
20
Under HBase policy we can edit/create HBase policies, this page shows how to create a policy at user level and
provide appropriate permissions.

21. Access error in Audit
21
Under Audit tab admin can view which user tried to access which table here user nabadeep got access denied as it did
not had the permission to put data in table testranger.

22. Audit Logs

23. Audit logs in JSON format
For each of the service like HDFS, Hive there will audit logs generated if enabled in
Ambari
23

24. Audit logs in JSON format
24

25. HDFS Audit File structure
25

26. Audit Log Storage Options
HDFS
Long term storage that can be used to understand user event trends and predict anomaly
RDBMS
MySQL, Oracle, Postgres, SQL Server
Solr
Good for quick reporting metrics to understand user event trends
Log4j Appenders

27. Best practices to use HDFS in Ranger
27
• Change HDFS umask to 077
fs.permissions.umask.mode=077
• IdenLfy directory which can be managed by Ranger policies
/apps/hive, /apps/Hbase
• IdenLfy directories which need to be managed by HDFS naLve permissions
/tmp and /user to 700
• Enable Ranger policy to audit all records

28. Best practices to use Hive in Ranger
28
• HiveServer2 access with limited HDFS access
̶ Column level access control over Hive data
• Hiveserver2, and HDFS files through Pig/MR jobs
̶ hive.server2.enable.doAs is set to "true“
• Hive CLI access

29. Atlas & Ranger

30. Tag Based Policies in Atlas
Ø Atlas and Ranger combination supports automation for governance and policies
Ø Atlas is where tags get set on metadata for example, a Customer table in Hive
can be tagged with value “PII”
Ø Ranger policies can be created on these tags to enforce access
Ø Ranger shows audit logs on access
Source: https://cwiki.apache.org/confluence/display/RANGER/Tag+Based+Policies

31. Ranger Tag based policy flow

32. Tag Service Setup – Ranger Admin
Source: https://cwiki.apache.org/confluence/display/RANGER/Tag+Based+Policies

33. Tag Policy Setup
Source: https://cwiki.apache.org/confluence/
display/RANGER/Tag+Based+Policies

34. Tag Policy Expiry

35. Backup

36. References
http://www.slideshare.net/trihug/trihug-october-apache-ranger
http://www.slideshare.net/RommelGarcia2/apache-ranger
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
http://hortonworks.com/blog/best-practices-for-hive-authorization-using-apache-ranger-in-
hdp-2-2
https://cwiki.apache.org/confluence/display/RANGER/Tag+Based+Policies

37. Q&A
rajesh.nadipalli@gmail.com
@ranadipa