This document discusses how to secure data access on Amazon EMR clusters using AWS Lake Formation. Lake Formation provides fine-grained, column-level access control and integrates with identity providers like Active Directory for single sign-on. It uses the AWS Glue Data Catalog as the metadata store. The document demonstrates how to configure an Amazon EMR cluster to use Lake Formation for authentication and query execution, controlling permissions at the table and column level rather than on the underlying Amazon S3 storage.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spark access control on Amazon EMR with
AWS Lake Formation
Anoop Johnson
A N T 2 2 3 - R
Principal Software Engineer
Amazon Web Services

2. What you will learn
Background
Authentication
Query execution
Demo

3. Lake Formation: Secure once, access in many ways
Amazon
Athena
Amazon
Redshift
Amazon
EMR
AWS
Glue
Amazon
S3
Data
catalog
Permissions
Lake Formation
Admin

4. Control data access with grant and revoke

5. Permissions on tables and columns, not Amazon
Simple Storage Service (Amazon S3)

6. View and audit data access

7. Why integrate Amazon EMR with Lake Formation?
• Fine-grained, column-level access to databases and tables
• Allows shared multi-tenant clusters to securely access data
• Uses the AWS Glue Data Catalog as the metadata store
• Federated single sign-on from your enterprise identity system
• Active Directory (AD FS), Auth0, Okta, and many others
• Uses Security Assertion Markup Language (SAML) 2.0

8. Query execution overview
Amazon
S3
Lake
Formation
Amazon
EMR

9. Amazon EMR authentication
Amazon EMR cluster
User
Used for
impersonation
Lake
Formation

10. Query execution under the hood
Amazon EMR worker node
1
32
4
Amazon
S3
5
6
Lake
Formation

11. Amazon EMR: Supported applications
• AWS Glue Data Catalog
• Identity providers with support for SAML
• Applications
• Spark SQL
• Amazon EMR Notebooks and Zeppelin with Livy

12. Integrating Amazon EMR with Lake Formation
• Establish trust relationship between your corporate IdP and AWS
• Configure IAM roles for Lake Formation
• Configure Amazon EMR security features
• Launch an Amazon EMR Lake Formation-enabled cluster

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.