Secure HBase from HUG-NYC

  • 7,368 views
Uploaded on

How security features are being developed to provide strong access control in HBase.

How security features are being developed to provide strong access control in HBase.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,368
On Slideshare
0
From Embeds
0
Number of Embeds
6

Actions

Shares
Downloads
127
Comments
0
Likes
12

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Secure HBase Hadoop Group @ Trend Micro: Andrew Purtell, Gary Helmling, Joshua Ho, Eugene Koontz, Mingjie Lai
  • 2. Overview
    • Why?
      • User isolation (control over your data)
      • 3. Multi-tenancy: private and public cloud
    • What is it?
      • Client access to HBase is authenticated
      • 4. User data is private unless access has been granted
      • 5. Access to data can be granted at a table or per column family basis
    • What is it not?
      • Row-level or per value (cell)
      • 6. Push down of file ownership to HDFS
      • 7. Full Role Based Access Control
  • 8. Concepts
    • Authentication
      • Who are you?
    • Authorization
      • Can user A do action X within context Y?
    • Isolation
      • System-wide concern
      • 9. Requires enforcement of authorization internally
      • 10. Authorization useless if system leaks data in other ways
        • Observability of storage files
        • 11. Eavesdropping on data in transit
  • 12. Authentication
    • Need to be able to confirm identities
    • 13. Building on Secure Hadoop RPC
      • Client authentication based on Kerberos
        • allows servers to verify client credentials with trusted third party
      • Secure RPC based on SASL
        • can provide confidential communications via GSSAPI/Kerberos
        • 14. also supports DIGEST-MD5 authentication, allowing Hadoop delegation token use for MapReduce
  • 15. Authorization
    • Default deny policy – full user isolation
    • 16. Permissions
      • Read, Write, Execute, Create, Admin
    • Permission Grants
      • Principal: user or group
      • 17. Scope: table, optional column family
      • 18. Permissions
    • Built-in Roles
      • Superuser: full access
      • 19. Table owner: full access to table, plus delegation
  • 20. Isolation
    • Optional RPC encryption with SASL
    • 21. Secure Hadoop
      • HDFS permissions to restrict access
      • 22. Table HFiles owned by HBase system user
    • HBase mediates access to table data
      • column family granularity
      • 23. -ROOT- and .META. readable by all
        • potential leakage of row key data
  • 24. How?
    • Access Control Lists (ACLs)
    • 25. Coprocessors
    • 26. Permission Storage : .META. to ZK to RegionServers
    Logical Principal Table:ColumnFamily Permissions humphrey foo:* {READ,WRITE} george foo:family1 {READ,WRITE} @reports (group) foo:family1 {READ} .META. acl: Row Qualifier Value foo,,1286569... humphrey RW george,family1 RW @reports,family1 R
  • 27. How?
    • Access Control Lists (ACLs)
    • 28. Coprocessors
    • 29. Permission Storage : .META. to ZK to RegionServers
  • 30. How?
    • Access Control Lists (ACLs)
    • 31. Coprocessors
    • 32. Permission Storage : .META. to ZK to RegionServers
  • 33. Synchronizing ACLs
    • .META. To Zookeeper
    • 34. ZooKeeper to RegionServers
  • 35. Synchronizing ACLs
    • .META. To Zookeeper
    • 36. ZooKeeper to RSs
  • 37. Access Control Lists
    • Canonical Storage : .META.'s acl: column family
    • 38. Client interface
      • hbase shell 'grant' command:
    hbase> create 'foo', 'f1' ... hbase> grant 'herbert', 'RW', 'foo' ... hbase> scan '.META.' ... ROW COLUMN+CELL foo,,1286569... column=acl:herbert, timestamp=1286569..., value=RW ...
  • 39. Coprocessors
    • AccessController: the "checking" in permissions checking
      • Anatomy of a client request
    Component User Action Class Method 1 client U sends IPC call C: <U,'get',T> to regionserver. HTable get() 2 regionserver S Receives C:<U,'get',T>. HBaseServer processData() 3 regionserver S -> U U.doAs('get',T) run() 4 regionserver U Check in-memory Permission Mirror: UserPerms = getUserPermissions(UserGroupInformation.getCurrentUser(),T) AccessController preGet() 5 regionserver U if (UserPerms imply Get) then return; else throw AccessDeniedException; 6 client U Receives either get() return value or AccessDeniedException HTable get()
  • 40. Next Steps
    • Handling of Master Operations
      • Create and Admin permissions
    • Delegation token authentication for MapReduce
    • 41. Additional permissions
      • Execute, Create, Admin
    • Roles
      • Current implementation is really ACL not RBAC
    • ZooKeeper Kerberos auth plugin
    • 42. Audit logging
    • 43. More granular access control
      • Per row or per KV?
      • 44. Would be implemented via meta-columns
  • 45. For More Information
    • HBase blog: Secure HBase by Eugene Koontz
    • 46. HBase JIRA
      • HBASE-1697: Discrentionary access control (umbrella issue)
      • 47. HBASE-3025: Coprocessor based access control
    • Code
      • http://github.com/trendmicro/hbase/tree/security