Security has always been a fundamental requirement for enterprise adoption. For example, in a company, billing, data science, and regional marketing teams may all have the required access privileges to view customer data, while sensitive data like credit card numbers should be accessible only to the finance team. Previously, Apache Hive™ with Apache Ranger™ policies was used to manage such scenarios. In this talk, we shows that Apache Spark™ SQL is aware of the existing Apache Ranger policies defined for Apache Hive. In other words, for SQL users, access to databases, tables, rows and columns are controlled in a fine-grained manner, irrespective of whether the data is analyzed using Apache Spark SQL or Hive. If a policy is updated, both Apache Spark and Apache Hive users get their result consistently. In addition, all fine-grained access via Apache Spark SQL can be monitored and searched through a centralized interface via Apache Ranger.

1. SECURITY UPDATES:
More Seamless Access Controls with
Apache Spark and Apache Ranger
Dongjoon Hyun @ Hortonworks Spark Team
Jason Dere @ Hortonworks Hive Team
June 2017

2. SECURITY UPDATES:
More Seamless Access Controls with
Apache Spark and Apache Ranger
Dongjoon Hyun

3. 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Agenda
Security Issues
Goals
Components
How it works
Demo

4. 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Background – Security
 One of fundamental features for enterprise adoption
– Multi-tenancy: Billing team / Data science team / Marketing teams
 Row and column-level access control for SQL users
– Row filtering
– Column masking
 Must enforce shared policies to various SQL engines simultaneously
– E.g. Apache Spark 2.1/1.6 and Apache Hive 2.1

5. 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Issue 1
 Spark reads all or nothing
 Directory/file-based permissions are insufficient for fine-grained
access control
Apache Spark is a general data processing engine
scala> val textFile = sc.textFile(“/apps/hive/warehouse/…")
textFile: org.apache.spark.rdd.RDD[String] = …

6. 6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Issue 2
 Permission 777 on warehouse?
Security starts from storage
Bad
Good

7. 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Issue 3
 New policies for SparkSQL?
 Rewrite Spark apps?
– Special data source tables
 Duplicated data maintained manually
– Filtered rows
– Removed or masked columns
Overhead during starting and maintaining security policies

8. 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Agenda
Security Issues
Goals
Components
How it works
Demo

9. 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Goal 1: Spark SQL Apps
Support row/column-level security with the batch apps
from pyspark.sql import SparkSession
spark = SparkSession
.builder
.enableHiveSupport()
.getOrCreate()
spark.sql("SELECT * FROM db_common.t_customer").show()
db_common
t_customer
…

10. 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Goal 2: Spark shells (1/2)
Support row/column-level security in all shells
spark-shell
pyspark

11. 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Goal 2: Spark shells (2/2)
Support row/column-level security in all shells
sparkR
spark-sql

12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Goal 3: Spark Thrift Server
Support row/column-level security with Spark Thrift Server
Login as `billing`
Login as `datascience`

13. SECURITY UPDATES:
More Seamless Access Controls with
Apache Spark and Apache Ranger
Jason Dere @ Hortonworks Hive Team

14. 14 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Agenda
Security Issues
Goals
Components
How it works
Demo

15. 15 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
What are required?
 Apache Ranger
 Apache Hive with LLAP
 Spark-LLAP (Apache License)
– A library and patches to integrate above tech with SparkSQL

16. 16 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Apache Ranger
Provide a standard authorization method across many Hadoop components
https://hortonworks.com/apache/ranger/#section_2

17. 17 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Ranger Policies – Column Access

18. 18 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Ranger Policies – Column Masking

19. 19 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Ranger Policies – Row Filtering

20. 20 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
YARN Cluster
HiveServer2
Client App
Hive Query
Coordinator
SQL Query:
select name from users
1
Apache Hive with LLAP
5
3 4
1.Client sends query to HiveServer2.
2.Query plan generation by HiveServer2.
3.Query plan sent to query coordinator
4.Query plan sent to LLAP daemons for
execution.
5.Results consolidated and sent to client
Plan Generation
TableScan: users
Projection: name
2
LLAP
LLAP
LLAP Daemons

21. 21 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Hive Security with Ranger
 Seamless integration with Ranger user-level access policies
– Column/row based security policies are applied automatically
– Hive query plans rewritten to apply masking/filtering functions on top of
the base table data.

22. 22 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
YARN Cluster
HiveServer2
Client App
Hive Query
Coordinator
SQL Query:
select name from users
1
HiveServer2 + LLAP
5
3 4
1.Client sends query to HiveServer2.
2.Query plan generation by HiveServer2.
3.Query plan sent to query coordinator
4.Query plan sent to LLAP daemons for
execution.
5.Results consolidated and sent to client
Plan Generation
TableScan: users
Projection: name
2
LLAP
LLAP
LLAP Daemons

23. 23 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
YARN Cluster
HiveServer2
Client App
Hive Query
Coordinator
Plan Generation
TableScan: users
Filter: state = ‘CA’
Projection: mask(name)
SQL Query:
select name from users
1.Client sends query to HiveServer2.
2.Query plan generation by HiveServer2.
Ranger security policies applied. Plan
modified based on dynamic security policies.
3.Query plan sent to query coordinator
4.Query plan sent to LLAP daemons for
execution. Filtering/masking performed.
5.Results consolidated and sent to client
1
HiveServer2 + LLAP + Ranger
Ranger
Dynamic Policies
5 2
3 4
LLAP
LLAP
LLAP Daemons

24. 24 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
External LLAP Client
 LLAP Daemon
– Persistent daemons combining query execution and in-memory caching
– External applications also able to use LLAP to retrieve data
• Provide a secure relational datanode view of the data

25. 25 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
LLAP
LLAP
LLAP Daemons
YARN Cluster
HiveServer2
Hive Query
Coordinator
Plan Generation
TableScan: users
Projection: name
1.Client requests data locations known as
“splits” from HiveServer2.
2.Query plan generation by HiveServer2.
3.Splits returned to client which include signed
query plan.
4.LLAP splits used by client to securely submit
query plan to LLAP. Data returned to client.
1
External LLAP Client
3 2
4
Client App
LLAP
InputFormat
SQL Query:
select name from users

26. 26 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
YARN Cluster
HiveServer2
Client App
Hive Query
Coordinator
Plan Generation
TableScan: users
Filter: state = ‘CA’
Projection: mask(name)
1.Client requests data locations known as
“splits” from HiveServer2.
2.Query plan generation by HiveServer2.
Ranger security policies applied. Plan
modified based on dynamic security policies.
3.Splits returned to client which include signed
query plan.
4.LLAP splits used by client to securely submit
query plan to LLAP. Filtering/masking
performed. Data returned to client.
1
External LLAP Client + Ranger
Ranger
Dynamic Policies
3 2
LLAP
InputFormat
SQL Query:
select name from users
LLAP
LLAP
LLAP Daemons
4

27. 27 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Agenda
Security Issues
Goals
Components
How it works
Demo

28. 28 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Spark-LLAP
 Spark connector library + patches on top of Spark
 Table data read securely through LLAP
 Leverages standard Ranger policies to control per-user
access/masking/filtering of data

29. 29 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Spark-LLAP: Credentials
 HDFS Delegation Token
– HDFSCredentialProvider gets it from namenode
 Hive Metastore Delegation Token
– HiveCredentialProvider gets it from Hive Metastore
 HiveServer2 Delegation Token
– HiveServer2CredentialProvider gets it from HiveServer2
Get and renew delegation tokens
Spark-LLAP
Existing

30. 30 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Spark-LLAP: LlapMetastoreCatalog
LlapMetastoreCatalog: Replaces MetastoreRelation with LlapRelation
SELECT gender, count(*)
FROM db_common.t_customer
WHERE name LIKE '%Obama’
GROUP BY gender
LlapRelation
SubqueryAlias
Analyzed Logical Plan
Filter: name like %Obama
Aggregate: gender
UnresolvedRelation
Filter: name like %Obama
Parsed Logical Plan
Aggregate: gender

31. 31 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Spark-LLAP: LlapMetastoreCatalog
LlapMetastoreCatalog: Replaces MetastoreRelation with LlapRelation
Without Spark-LLAP
With Spark-LLAP

32. 32 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
YARN Cluster
HiveServer2
LlapRelation
Hive Query
Coordinator
Plan Generation
TableScan: users
Filter: state = ‘CA’
Projection mask(name)
1
Spark-LLAP: LlapRelation
Ranger
Dynamic Policies
3 2
LLAP
InputFormat
SQL Query:
select name from users
LLAP
LLAP
LLAP Daemons
4
Uses LLAP external client API to read table data

33. 33 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Spark-LLAP: LlapRelation
LlapRelation supports predicate pushdown and column pruning
LlapRelation
SubqueryAlias
Analyzed Logical Plan
Filter: name like %Obama
Aggregate: gender
LlapRelation
Filter: EndsWith(name,Obama)
Optimized Logical Plan
Project: gender
Aggregate: gender
Scan LlapRelation
PushedFilter: StringEndsWith(…)
ReadSchema: gender
Filter: EndsWith(name, Obama)
Physical Plan
Project: gender
HashAggregate: gender
…

34. 34 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Using Spark-LLAP
 spark-submit
--package spark-llap.jar
--conf spark.sql.hive.llap=true
--conf spark.yarn.security.credentials.hiveserver2.enabled=true
--master yarn
--deploy-mode cluster
sql.py
Launch Spark jobs `--package` option is supported, too
Easy to turn on/off
Only used for YARN cluster mode

35. 35 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Agenda
Security Issues
Goals
Components
How it works
Demo

36. 36 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Spark-LLAP for Spark 1.6 (TP)
• Use Ranger for SELECT statement
• Use LlapContext
HDP 2.5.X
Milestone
Spark-LLAP for Spark 2.1.0 (TP)
• Use Ranger for more statements (in STS)
• No need to rewrite codes
• Support all languages and shells
HDP 2.6.0 HDP 2.6.1
Spark-LLAP for Spark 2.1.1 (TP)
• Support YARN cluster mode
• Support Hive complex types
Spark-LLAP for Spark 2.2.0
• Available soon in GitHub
HDP X.X.X

37. 37 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Resources
 GitHub
– https://github.com/hortonworks-spark/spark-llap
 Maven
– http://repo.hortonworks.com/content/groups/public/com/hortonworks/spark/spark-
llap_2.11/
 Youtube Demo
– https://www.youtube.com/watch?v=_-oYpQGWm5k (HDP 2.6.1)
 Hortonworks Blog
– https://hortonworks.com/blog/row-column-level-control-apache-spark/
 Hortonworks Community Connection Article
– https://community.hortonworks.com/articles/101181/rowcolumn-level-security-in-sql-for-
apache-spark-2.html
 Support Matrix
– https://github.com/hortonworks-spark/spark-llap/wiki/7.-Support-Matrix

38. 38 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Summary
 Support row/column-level security with
– Spark apps with YARN client/cluster mode
– Spark shells
– Spark Thrift Server
 You can use the existing Spark 2.X SQL apps and scripts
 Easy to turn on/off with only configurations
 Ranger enforces Hive/Spark simultaneously and consistently
Spark-LLAP with HDP 2.6.1 is TP

39. 39 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Acknowledgement
 Apache Hive / Apache Spark / Apache Ranger Community
 Bikas Saha, Mingjie Tang, Saisai Shao, Siddharth Seth, Sergey
Shelukhin, Thejas Nair, Zhan Zhang, and many others

40. 40 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Thank you