Apache Argus is a centralized security framework for Hadoop that provides authentication, authorization, auditing, and data protection. It aims to manage security across the entire Hadoop ecosystem through a single point of administration. Key features include role-based authorizations with fine-grained access control for HDFS, Hive, and HBase resources, configurable auditing, and a portal for centralized policy management and reporting. The framework uses plugins to integrate with Hadoop components and supports REST APIs for programmatic access.

1. Apache Argus
Comprehensive Security for Enterprise Hadoop
Page 1 © Hortonworks Inc. 2014
Don Bosco Durai
Balaji Ganesan

2. Agenda
• Security Landscape
• Apache Argus Vision
• Apache Argus Features
• Apache Argus Architecture
• Demo
Page 2 © Hortonworks Inc. 2014

3. Security needs are changing
Administration
Centrally management &
consistent security
Authentication
Authenticate users and systems
Authorization
Provision access to data
Audit
Maintain a record of data access
Data Protection
Protect data at rest and in motion
Page 3 © Hortonworks Inc. 2014
Security needs are changing
• YARN unlocks the data lake
• Multi-tenant: Multiple applications for data access
• Changing and complex compliance environment
• ETL of non-sensitive data can yield sensitive data
Summer 2014
65% of clusters host
multiple workloads Fall 2013
Largely silo’d deployments
with single workload clusters
5 areas of security focus

4. Apache Argus Vision
• Provide a centralized framework for managing security within Hadoop
ecosystem
• Single point of administration for authentication, authorization, auditing and data protection
• Integrated approach for entire Hadoop ecosystem, i.e., platform level
security
• Tight integration with components
• Integrate with standard APIs, hooks for access control
• Maintain single version of truth, for consistency in access policies
• Extensible Infrastructure
• REST APIs for managing policies
• Pluggable enforcement
Page 4 © Hortonworks Inc. 2014

5. Security in Hadoop w/ Apache Argus
Centralized Security Administration
• Fine grain
encryption*
• Enabled
partners* Hadoop Apache Argus
Page 5 © Hortonworks Inc. 2014
Authorization
Restrict access to
explicit data
Audit
Understand who
did what
Data Protection
Encrypt data at
rest & in motion
• Kerberos in native
Apache Hadoop
• HTTP/REST API
Secured with
Apache Knox
Gateway
• HDFS Permissions, HDFS Extended
ACLs
• Audit logs in with HDFS & MR
• Hive ATZ-NG
Authentication
Who am I/prove it?
• Wire encryption
in Hadoop
• Orchestrated
encryption with
3rd party tools
• HDFS, Hive ,
Hbase, Storm,
Knox, etc.
• Fine grain
access control
• RBAC
• Centralized
audit reporting
• Policy and
access history
• SSO*
• Provisioning*
• Brokering*

6. Apache Argus Security Features - Current
Page 6 © Hortonworks Inc. 2014
Apache Argus
Role Based Authorizations
Fine grained access control HDFS – Folder, File,
Hive – Database, Table, Column, UDFs
HBase – Table, Column Family, Column
Wildcard Resource Names Yes
Permission Support HDFS – Read, Write, Execute
Hive – Select, Update, Create, Drop, Alter, Index, Lock
Hbase – Read, Write, Create
Auditing
Configurable audit Auditing can be controlled through policy
Resource access auditing User id, request type, repository, access resource, IP
address, timestamp, access granted/denied
Admin auditing Changes to policies, login sessions and agent
monitoring,

7. Apache Argus Security Features
Page 7 © Hortonworks Inc. 2014
Apache Argus
Reporting
Basic reporting in portal Global view of policies, across HDFS, Hbase and
Hive
Manage
User/ Group mapping Local, Sync with LDAP/AD, Sync with Unix
Delegated administration Delegate policy administration to groups or users

8. Central Security Administration
Argus - Portal
• Delivers a ‘single pane of glass’ for the security administrator
• Centralizes administration of security policy
Page 8 © Hortonworks Inc. 2014

9. Setup Authorization Policies - HDFS
Page 9 © Hortonworks Inc. 2014
file level
access
control,
flexible
definition
Control
permissions

10. Setup Authorization Policies - Hive
Page 10 © Hortonworks Inc. 2014

11. Monitor through Auditing
11
Page 11 © Hortonworks Inc. 2014

12. Authorization and Auditing w/ Argus
Hadoop distributed
file system (HDFS)
Page 12 © Hortonworks Inc. 2014
Argus Administration Portal
HBase
Hive Server2
Argus Policy
Server
Argus Audit
Server
Argus
Plugin
Hadoop Components Enterprise
Users
Argus
Plugin
Argus
Plugin
Legacy
Tools
Integration API
RDBMS
HDFS
Knox
[components]
Argus
Plugin*
Argus
Plugin*
Argus
Plugin*
Storm
YARN
:
Data
Opera.ng
System
* - Future Integration

13. Resources
Page 13 © Hortonworks Inc. 2014
Page 13

14. Argus Page
Page 14 © Hortonworks Inc. 2014

15. Argus GitHub
Page 15 © Hortonworks Inc. 2014

16. Roadmap
Page 16 © Hortonworks Inc. 2014
Page 16

17. Hortonworks Security Investment Plans
HDP + Apache Argus
Comprehensive Security for Enterprise Hadoop
Comprehensive Security
Meet all security requirements across Authentication,
Authorization, Audit & Data Protection for all HDP
components
Page 17 © Hortonworks Inc. 2014
…all IN Hadoop
Goals:
Investment
themes
Central Administration
Provide one location for administering security policies and
audit reporting for entire platform
Consistent Integration
Integrate with other security & identity management systems,
for compliance with IT policies
Previous
Phases
ü Kerberos
Authen@ca@on
ü HDFS,
Hive
&
Hbase
authoriza@on
ü Wire
Encryp@on
for
data
in
mo@on
ü Knox
for
perimeter
security
ü SQL
Style
Hive
Authoriza@on
ü ACLs
for
HDFS
Current
Phase
• Centralized
Delivered XA
Secure
Security
Admin
for
HDFS,
Hive
&
Hbase,
Storm,
Knox
• Centralized
Audit
Repor@ng
• Delegated
Policy
Administra@on
Delivered
Future
Phases
• Encryp@on
in
HDFS,
Hive
&
Hbase
• Centralized
security
administra@on
of
en@re
Hadoop
plaHorm
• Centralized
audi@ng
of
en@re
plaHorm
• Expand
Authen@ca@on
&
SSO
integra@on
choices
• Tag
based
global
policies
(e.g.
Policy
for
PII)

18. Page 18 © Hortonworks Inc. 2014
Q&A
Page 18