SlideShare a Scribd company logo
1 of 33
hosted by
Kerberos-based Big Data Security
Solution and Practice in Alibaba
Cloud HBase
Jiajia Li @ Intel
Chao Guo @ Alibaba
August 17,2018
hosted by
Content
01
02
Hadoop Authentication Service
Security Practice in ApsaraDB
for HBase
hosted by
01
Hadoop Authentication Service
Jiajia Li Intel
hosted by
Content
1.1
1.2
1.3
Background
Introduction to HAS
Outlook and Summary
hosted by
Background1.1
hosted by
Background
Motivations
• Jan 2017, The ransomware
attacks on poorly secured
MongoDB (Over 27000
MongoDB Databases Held For
Ransom Within A Week), cyber
crooks have started targeting
unprotected Hadoop Clusters as
well (Hadoop, CounchDB Next
Targets in Wave of Database
Attacks), until now, large
amounts of Hadoop cluster data
is still exposed in the public
network (Insecure Hadoop
Clusters Expose Over 5,000
Terabytes of Data).
hosted by
Background
Multiple Ways to Attack Insecure Hadoop Cluster
Hadoop* Cluster
Service
I’m Datanode
HDFS with ACL
superuser: hadoop
hadoop fs -rmr /
login with
hadoop
User
hosted by
Background
How to Secure a Hadoop Cluster
Authentication Authorization
 Authentication
• Kerberos is the right approach adopted for Hadoop security
• MIT Kerberos, Azure AD, HAS
 Authorization
• Apache Sentry(Cloudera), Apache Ranger(Hortonworks)
hosted by
Introduction to HAS1.2
hosted by
Introduction to HAS
Challenges for the Existing Solution
Hard to integrate existing identity management systems of enterprises to Kerberos
Over the past few years, multiple cloud providers have introduced Hadoop-as-a-Service and
more organizations consider the cloud as a component of their Hadoop deployments
Java lacks a comprehensive Kerberos library. The Kerberos support in Java/JRE is:
• Limited, lacking full encryption and checksum types
• Hidden from GSSAPI/SASL layers
• Evolving slow
Very difficult in Kerberos cluster deployment
Kerberos is essentially a protocol, or secure channel, doesn’t have to be that complex to
most or normal users, hiding the details
hosted by
Introduction to HAS
HAS System Architecture
Active
Directory
LDAP
RDMS
OAuth
SSO (SAML)
PKI
OTP
HAS Server
Hadoop
Kerberos
KDC
(Kerby)
Token
Authority
Kerberos
Client
HAS Client
1
2
3 4
5
6
Enterprise
Identity
HAS Token
Kerberos Token
HAS is a secure and
extensible authentication
framework for addressing
the problem of integration
of enterprise identity with
Kerberos centric Hadoop
ecosystem.
hosted by
Introduction to HAS
Key points of HAS implementationt
Based on the new authentication
mechanism, security administrators
don’t need to synchronize user
account information to the Kerberos
database.
Hadoop users can also continue
to log in using a familiar
authentication mode.
In the new authentication mechanism,
the combination of your plugin and
the existing authentication system
can be customized and implemented.
Hadoop services continuously use
the original Kerberos authentication
mechanism.
hosted by
Introduction to HAS
HAS protocol flow
hosted by
Introduction to HAS
HAS plugins
hosted by
Outlook and Summary1.3
hosted by
Outlook and Summary
• HAS will be merged to trunk in the master JIRA (https://issues.apache.org/jira/browse/DIRKRB-671).
• According to the community plan, the HAS feature will be released in Kerby 2.0.0, and Kerby 2.0.0 will be
released in the near future.
• The new authentication mechanism (Kerberos-based Token Authentication) provided in HAS supports most
components in the Hadoop ecosystem and makes little or no change to the components.
• HAS open source in the branch of Apache Kerby project (https://github.com/apache/directory-
kerby/tree/has-project).
hosted by
Security Practice in ApsaraDB for
HBase
Chao guo Aliyun
02
hosted by
Content
2.1
2.2
Introduction to Apache HBase Security
and Security of ApsaraDB for HBase
ApsaraDB for HBase Optimization
base on HAS
2.3 Outlook and summary
hosted by
Introduction to Apache HBase Security and
Security of ApsaraDB for HBase2.1
hosted by
DN NN
HDFS
zookeeper
Apach HBase
ACL SASL RPC HTTPS
Introduction to Apache HBase Security and
Security of ApsaraDB for HBase
Auth client Https WEB
User
Apache HBase Security includes :
1. Access control Label base on Access Controller
coprocessor;
2. Using Secure HTTP for Web UI.
3. Kerberos authentication for RPC;
Introduce to Apache HBase Security
hosted by
Introduction to ApsaraDB for Hbase security
Public networker user Classic network user
Introduction to Apache HBase Security and
Security of ApsaraDB for HBase
White list
text
NetworkIsolation
User VPC/Classic
Plugin authentication Other mechanism
HAS
HTTPSACL
AUDIT QUOTA
Authentication
ENCRYPT
ApsaraDB for HBase
ApsaraDB for HBase main function:
• Network Isolation and white list;
• HAS authentication;
• …
hosted byIntroduction to Apache HBase Security
and Security of ApsaraDB for HBase
text
• Instal MIT kerberos at the locality;
• Set up local krb5.conf,the Kerberos service address,realm and so on of the HBase;
• User name and local system user name should be the same, If got no user, need
create;
• Set up security configuration in hbase-site.xml,core-site.xml,hdfs-site.xml.For
example:hadoop.security.authentication,dfs.namenode.kerberos.principal;
• Do kinit, run kinit throw user passwor mode or keytab, get the legal user tiket;
• Access to hbase throw hbase shell .
VS
• Set up hbase zookeeper address, user’s password and name;
• Access to hbase throw hbase shell .
Introduction to ApsaraDB for HBase Autnentication
hosted byIntroduction to Apache HBase Security
and Security of ApsaraDB for HBase
Security
network isolation/whitelist/Authentication
Cost
Reduce the cost of operation and maintenance,
perfermance
Affinity
Simplified configuration;easy to use
Userexperience
User’s needs start basically from security then low
cost then good experience
User experience
hosted by
ApsaraDB for HBase Optimization base on
HAS2.2
hosted by
ApsaraDB for HBase Optimization base on HAS
Basical Introduction
Why choice HAS for us?
• Kerberos is the only means to enable hadoop security
• Compatible with existting security mechanism :ACL、
keberos …
• Easy to deploy
• Easy to use for client
• Good scalability
• Performance is ok
• Low operating cost;
• …
hosted by
ApsaraDB for HBase Optimization base on HAS
High availability backend
Impement zookeeper like backend for storage of user name/password/whitelist
host/kdcconf
Account password management
Using plugin for password and account management
Security and practice
Whitelist for hosts and other securities
ApsaraDB for HBase’s authentication improvement
hosted by
ApsaraDB for HBase Optimization base on HAS
Security and practice
• HA for HAS server
• White list for host access
• Plugins use
HTTPS(Initialiazation/usage)
• Configurable salting
algorithm
• Backward compatible
kerberos and all Hbase
security
• Ops tools : one button to
deploy
hosted by
ApsaraDB for HBase Optimization base on HAS
Account password management
Aliyun Client/Server plugin mode
• Client can initialize with their user/password/hosts
• Client pass user/password throw https
• Server plugin verify from storage with salt
• The entire process is safe
• Configure free,easy to use;
Other plugin mode:
We have done some service on has like:
hosted by
High availability backend
Mysql not fit for cloud environment
Single mysql is not ha , 2 or 3 node ha mysql not fit for cloud
environment deployment , k-v format, zk is enough
At least 3 replicas ,more available than mysql
Most value is less than 50B , less memory than mysql and
enough performance
ApsaraDB for HBase Optimization base on HAS
High availability for backend storage
Less resource and enough performance
hosted by
ApsaraDB for HBase Optimization base on HAS
Possible user case
User consulation case:
• All users with same export Ip
• ACL is not suitable
• Users need different authentication
On Cloud almost one small cluster
for on user;
But private cloud and big cluster is
different;
hosted by
Outlook and Summary2.3
hosted by
Outlook and Summary
• For the security of ApsaraDB for HBase, we got network ioslation 、whitelist and other mechanism . As
authentication ,we adopt HAS.
• We have do some optimization for HAS such as ha、customized plugins.
• The secure ApsaraDB for HBase is ready now.
https://www.aliyun.com/product/hbase?spm=5176.8142029.388261.280.702b614asnEokD
hosted by
Thanks
Personal WechatDingding group for HBase

More Related Content

What's hot

HBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBase
HBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBaseHBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBase
HBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBase
Michael Stack
 

What's hot (20)

HBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBase
HBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBaseHBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBase
HBaseConAsia2018: Track2-5: JanusGraph-Distributed graph database with HBase
 
HBaseConAsia2018 Track3-5: HBase Practice at Lianjia
HBaseConAsia2018 Track3-5: HBase Practice at LianjiaHBaseConAsia2018 Track3-5: HBase Practice at Lianjia
HBaseConAsia2018 Track3-5: HBase Practice at Lianjia
 
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
 
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life InsuranceHBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
 
HBaseConAsia2018 Track1-1: Use CCSMap to improve HBase YGC time
HBaseConAsia2018 Track1-1: Use CCSMap to improve HBase YGC timeHBaseConAsia2018 Track1-1: Use CCSMap to improve HBase YGC time
HBaseConAsia2018 Track1-1: Use CCSMap to improve HBase YGC time
 
HBaseConAsia2018 Track3-6: HBase at Meituan
HBaseConAsia2018 Track3-6: HBase at MeituanHBaseConAsia2018 Track3-6: HBase at Meituan
HBaseConAsia2018 Track3-6: HBase at Meituan
 
HBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
HBaseCon 2012 | Building a Large Search Platform on a Shoestring BudgetHBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
HBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
 
HBaseConAsia2018 Track1-3: HBase at Xiaomi
HBaseConAsia2018 Track1-3: HBase at XiaomiHBaseConAsia2018 Track1-3: HBase at Xiaomi
HBaseConAsia2018 Track1-3: HBase at Xiaomi
 
HBaseConAsia2018 Keynote1: Apache HBase Project Status
HBaseConAsia2018 Keynote1: Apache HBase Project StatusHBaseConAsia2018 Keynote1: Apache HBase Project Status
HBaseConAsia2018 Keynote1: Apache HBase Project Status
 
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
 
HBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at DidiHBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at Didi
 
Hadoop @ eBay: Past, Present, and Future
Hadoop @ eBay: Past, Present, and FutureHadoop @ eBay: Past, Present, and Future
Hadoop @ eBay: Past, Present, and Future
 
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index StructuresHBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures
 
HBaseCon 2015: State of HBase Docs and How to Contribute
HBaseCon 2015: State of HBase Docs and How to ContributeHBaseCon 2015: State of HBase Docs and How to Contribute
HBaseCon 2015: State of HBase Docs and How to Contribute
 
Large-scale Web Apps @ Pinterest
Large-scale Web Apps @ PinterestLarge-scale Web Apps @ Pinterest
Large-scale Web Apps @ Pinterest
 
HBaseCon 2015- HBase @ Flipboard
HBaseCon 2015- HBase @ FlipboardHBaseCon 2015- HBase @ Flipboard
HBaseCon 2015- HBase @ Flipboard
 
HBaseCon 2013: Rebuilding for Scale on Apache HBase
HBaseCon 2013: Rebuilding for Scale on Apache HBaseHBaseCon 2013: Rebuilding for Scale on Apache HBase
HBaseCon 2013: Rebuilding for Scale on Apache HBase
 
HBaseCon 2015 General Session: State of HBase
HBaseCon 2015 General Session: State of HBaseHBaseCon 2015 General Session: State of HBase
HBaseCon 2015 General Session: State of HBase
 
HBaseCon 2015: Graph Processing of Stock Market Order Flow in HBase on AWS
HBaseCon 2015: Graph Processing of Stock Market Order Flow in HBase on AWSHBaseCon 2015: Graph Processing of Stock Market Order Flow in HBase on AWS
HBaseCon 2015: Graph Processing of Stock Market Order Flow in HBase on AWS
 
Harmonizing Multi-tenant HBase Clusters for Managing Workload Diversity
Harmonizing Multi-tenant HBase Clusters for Managing Workload DiversityHarmonizing Multi-tenant HBase Clusters for Managing Workload Diversity
Harmonizing Multi-tenant HBase Clusters for Managing Workload Diversity
 

Similar to HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Practice in Alibaba Cloud HBase

Hadoop Security Today and Tomorrow
Hadoop Security Today and TomorrowHadoop Security Today and Tomorrow
Hadoop Security Today and Tomorrow
DataWorks Summit
 
Improvements in Hadoop Security
Improvements in Hadoop SecurityImprovements in Hadoop Security
Improvements in Hadoop Security
DataWorks Summit
 

Similar to HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Practice in Alibaba Cloud HBase (20)

Improvements in Hadoop Security
Improvements in Hadoop SecurityImprovements in Hadoop Security
Improvements in Hadoop Security
 
Troubleshooting Kerberos in Hadoop: Taming the Beast
Troubleshooting Kerberos in Hadoop: Taming the BeastTroubleshooting Kerberos in Hadoop: Taming the Beast
Troubleshooting Kerberos in Hadoop: Taming the Beast
 
HBaseCon 2012 | HBase Security for the Enterprise - Andrew Purtell, Trend Micro
HBaseCon 2012 | HBase Security for the Enterprise - Andrew Purtell, Trend MicroHBaseCon 2012 | HBase Security for the Enterprise - Andrew Purtell, Trend Micro
HBaseCon 2012 | HBase Security for the Enterprise - Andrew Purtell, Trend Micro
 
Hadoop Security Today and Tomorrow
Hadoop Security Today and TomorrowHadoop Security Today and Tomorrow
Hadoop Security Today and Tomorrow
 
Securing the Hadoop Ecosystem
Securing the Hadoop EcosystemSecuring the Hadoop Ecosystem
Securing the Hadoop Ecosystem
 
Hadoop Security Today & Tomorrow with Apache Knox
Hadoop Security Today & Tomorrow with Apache KnoxHadoop Security Today & Tomorrow with Apache Knox
Hadoop Security Today & Tomorrow with Apache Knox
 
2014 sept 4_hadoop_security
2014 sept 4_hadoop_security2014 sept 4_hadoop_security
2014 sept 4_hadoop_security
 
Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon
Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheConTechnical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon
Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon
 
Improvements in Hadoop Security
Improvements in Hadoop SecurityImprovements in Hadoop Security
Improvements in Hadoop Security
 
Hdp security overview
Hdp security overview Hdp security overview
Hdp security overview
 
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding EdgeCIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
 
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive
 
Hadoop & Security - Past, Present, Future
Hadoop & Security - Past, Present, FutureHadoop & Security - Past, Present, Future
Hadoop & Security - Past, Present, Future
 
Apache Argus - How do I secure my entire Hadoop cluster? Olivier Renault @ Ho...
Apache Argus - How do I secure my entire Hadoop cluster? Olivier Renault @ Ho...Apache Argus - How do I secure my entire Hadoop cluster? Olivier Renault @ Ho...
Apache Argus - How do I secure my entire Hadoop cluster? Olivier Renault @ Ho...
 
Treat your enterprise data lake indigestion: Enterprise ready security and go...
Treat your enterprise data lake indigestion: Enterprise ready security and go...Treat your enterprise data lake indigestion: Enterprise ready security and go...
Treat your enterprise data lake indigestion: Enterprise ready security and go...
 
Introduction to HBase - NoSqlNow2015
Introduction to HBase - NoSqlNow2015Introduction to HBase - NoSqlNow2015
Introduction to HBase - NoSqlNow2015
 
AWS Public Sector Symposium 2014 Canberra | Secure Hadoop as a Service
AWS Public Sector Symposium 2014 Canberra | Secure Hadoop as a ServiceAWS Public Sector Symposium 2014 Canberra | Secure Hadoop as a Service
AWS Public Sector Symposium 2014 Canberra | Secure Hadoop as a Service
 
Apache HBase 0.98
Apache HBase 0.98Apache HBase 0.98
Apache HBase 0.98
 
Secure Hadoop clusters on Windows platform
Secure Hadoop clusters on Windows platformSecure Hadoop clusters on Windows platform
Secure Hadoop clusters on Windows platform
 
TriHUG October: Apache Ranger
TriHUG October: Apache RangerTriHUG October: Apache Ranger
TriHUG October: Apache Ranger
 

More from Michael Stack

More from Michael Stack (20)

hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloudhbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
 
hbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinteresthbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinterest
 
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltdhbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
 
hbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didihbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didi
 
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
 
hbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencenthbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencent
 
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
 
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
 
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Componenthbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
 
hbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomihbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomi
 
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Sparkhbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
 
hbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solutionhbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solution
 
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memoryhbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
 
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACLhbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
 
hbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBasehbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBase
 
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
 
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
 
HBaseConAsia2019 Keynote
HBaseConAsia2019 KeynoteHBaseConAsia2019 Keynote
HBaseConAsia2019 Keynote
 

Recently uploaded

一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样
一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样
一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样
AS
 
一比一原版英国格林多大学毕业证如何办理
一比一原版英国格林多大学毕业证如何办理一比一原版英国格林多大学毕业证如何办理
一比一原版英国格林多大学毕业证如何办理
AS
 
原版定制英国赫瑞瓦特大学毕业证原件一模一样
原版定制英国赫瑞瓦特大学毕业证原件一模一样原版定制英国赫瑞瓦特大学毕业证原件一模一样
原版定制英国赫瑞瓦特大学毕业证原件一模一样
AS
 
一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书
一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书
一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书
AS
 
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
AS
 
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
Fi
 
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
Fi
 
Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...
Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...
Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...
ZurliaSoop
 
一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样
一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样
一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样
ayvbos
 
Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...
Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...
Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...
mikehavy0
 
@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)
@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)
@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)
Obat Cytotec
 
一比一原版帝国理工学院毕业证如何办理
一比一原版帝国理工学院毕业证如何办理一比一原版帝国理工学院毕业证如何办理
一比一原版帝国理工学院毕业证如何办理
F
 

Recently uploaded (20)

一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样
一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样
一比一原版(毕业证书)新加坡南洋理工学院毕业证原件一模一样
 
(VIP) ℂall Girls Lucknow < Hire Me Sneha 📞8630512678 > Home Delivery For Full...
(VIP) ℂall Girls Lucknow < Hire Me Sneha 📞8630512678 > Home Delivery For Full...(VIP) ℂall Girls Lucknow < Hire Me Sneha 📞8630512678 > Home Delivery For Full...
(VIP) ℂall Girls Lucknow < Hire Me Sneha 📞8630512678 > Home Delivery For Full...
 
一比一原版英国格林多大学毕业证如何办理
一比一原版英国格林多大学毕业证如何办理一比一原版英国格林多大学毕业证如何办理
一比一原版英国格林多大学毕业证如何办理
 
Loker Pemandu Lagu LC Semarang 085746015303
Loker Pemandu Lagu LC Semarang 085746015303Loker Pemandu Lagu LC Semarang 085746015303
Loker Pemandu Lagu LC Semarang 085746015303
 
原版定制英国赫瑞瓦特大学毕业证原件一模一样
原版定制英国赫瑞瓦特大学毕业证原件一模一样原版定制英国赫瑞瓦特大学毕业证原件一模一样
原版定制英国赫瑞瓦特大学毕业证原件一模一样
 
一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书
一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书
一比一定制(Waikato毕业证书)新西兰怀卡托大学毕业证学位证书
 
The Rise of Subscription-Based Digital Services.pdf
The Rise of Subscription-Based Digital Services.pdfThe Rise of Subscription-Based Digital Services.pdf
The Rise of Subscription-Based Digital Services.pdf
 
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
 
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
 
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
 
APNIC Updates presented by Paul Wilson at CaribNOG 27
APNIC Updates presented by Paul Wilson at  CaribNOG 27APNIC Updates presented by Paul Wilson at  CaribNOG 27
APNIC Updates presented by Paul Wilson at CaribNOG 27
 
Abortion Pills In Jeddah+966572737505 & Get cytotec Jeddah
Abortion Pills In Jeddah+966572737505 & Get cytotec JeddahAbortion Pills In Jeddah+966572737505 & Get cytotec Jeddah
Abortion Pills In Jeddah+966572737505 & Get cytotec Jeddah
 
Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...
Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...
Jual obat aborsi Bekasi ( 085657271886 ) Cytote pil telat bulan penggugur kan...
 
一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样
一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样
一比一原版(USYD毕业证书)悉尼大学毕业证原件一模一样
 
APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...
APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...
APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...
 
Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...
Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...
Abortion Clinic in Germiston +27791653574 WhatsApp Abortion Clinic Services i...
 
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
 
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays SweatshirtsFree on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
 
@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)
@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)
@OBAT ABORSI 3 BULAN@ OBAT PENGGUGUR KANDUNGAN 3 BULAN (087776558899)
 
一比一原版帝国理工学院毕业证如何办理
一比一原版帝国理工学院毕业证如何办理一比一原版帝国理工学院毕业证如何办理
一比一原版帝国理工学院毕业证如何办理
 

HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Practice in Alibaba Cloud HBase

  • 1. hosted by Kerberos-based Big Data Security Solution and Practice in Alibaba Cloud HBase Jiajia Li @ Intel Chao Guo @ Alibaba August 17,2018
  • 2. hosted by Content 01 02 Hadoop Authentication Service Security Practice in ApsaraDB for HBase
  • 3. hosted by 01 Hadoop Authentication Service Jiajia Li Intel
  • 6. hosted by Background Motivations • Jan 2017, The ransomware attacks on poorly secured MongoDB (Over 27000 MongoDB Databases Held For Ransom Within A Week), cyber crooks have started targeting unprotected Hadoop Clusters as well (Hadoop, CounchDB Next Targets in Wave of Database Attacks), until now, large amounts of Hadoop cluster data is still exposed in the public network (Insecure Hadoop Clusters Expose Over 5,000 Terabytes of Data).
  • 7. hosted by Background Multiple Ways to Attack Insecure Hadoop Cluster Hadoop* Cluster Service I’m Datanode HDFS with ACL superuser: hadoop hadoop fs -rmr / login with hadoop User
  • 8. hosted by Background How to Secure a Hadoop Cluster Authentication Authorization  Authentication • Kerberos is the right approach adopted for Hadoop security • MIT Kerberos, Azure AD, HAS  Authorization • Apache Sentry(Cloudera), Apache Ranger(Hortonworks)
  • 10. hosted by Introduction to HAS Challenges for the Existing Solution Hard to integrate existing identity management systems of enterprises to Kerberos Over the past few years, multiple cloud providers have introduced Hadoop-as-a-Service and more organizations consider the cloud as a component of their Hadoop deployments Java lacks a comprehensive Kerberos library. The Kerberos support in Java/JRE is: • Limited, lacking full encryption and checksum types • Hidden from GSSAPI/SASL layers • Evolving slow Very difficult in Kerberos cluster deployment Kerberos is essentially a protocol, or secure channel, doesn’t have to be that complex to most or normal users, hiding the details
  • 11. hosted by Introduction to HAS HAS System Architecture Active Directory LDAP RDMS OAuth SSO (SAML) PKI OTP HAS Server Hadoop Kerberos KDC (Kerby) Token Authority Kerberos Client HAS Client 1 2 3 4 5 6 Enterprise Identity HAS Token Kerberos Token HAS is a secure and extensible authentication framework for addressing the problem of integration of enterprise identity with Kerberos centric Hadoop ecosystem.
  • 12. hosted by Introduction to HAS Key points of HAS implementationt Based on the new authentication mechanism, security administrators don’t need to synchronize user account information to the Kerberos database. Hadoop users can also continue to log in using a familiar authentication mode. In the new authentication mechanism, the combination of your plugin and the existing authentication system can be customized and implemented. Hadoop services continuously use the original Kerberos authentication mechanism.
  • 13. hosted by Introduction to HAS HAS protocol flow
  • 14. hosted by Introduction to HAS HAS plugins
  • 15. hosted by Outlook and Summary1.3
  • 16. hosted by Outlook and Summary • HAS will be merged to trunk in the master JIRA (https://issues.apache.org/jira/browse/DIRKRB-671). • According to the community plan, the HAS feature will be released in Kerby 2.0.0, and Kerby 2.0.0 will be released in the near future. • The new authentication mechanism (Kerberos-based Token Authentication) provided in HAS supports most components in the Hadoop ecosystem and makes little or no change to the components. • HAS open source in the branch of Apache Kerby project (https://github.com/apache/directory- kerby/tree/has-project).
  • 17. hosted by Security Practice in ApsaraDB for HBase Chao guo Aliyun 02
  • 18. hosted by Content 2.1 2.2 Introduction to Apache HBase Security and Security of ApsaraDB for HBase ApsaraDB for HBase Optimization base on HAS 2.3 Outlook and summary
  • 19. hosted by Introduction to Apache HBase Security and Security of ApsaraDB for HBase2.1
  • 20. hosted by DN NN HDFS zookeeper Apach HBase ACL SASL RPC HTTPS Introduction to Apache HBase Security and Security of ApsaraDB for HBase Auth client Https WEB User Apache HBase Security includes : 1. Access control Label base on Access Controller coprocessor; 2. Using Secure HTTP for Web UI. 3. Kerberos authentication for RPC; Introduce to Apache HBase Security
  • 21. hosted by Introduction to ApsaraDB for Hbase security Public networker user Classic network user Introduction to Apache HBase Security and Security of ApsaraDB for HBase White list text NetworkIsolation User VPC/Classic Plugin authentication Other mechanism HAS HTTPSACL AUDIT QUOTA Authentication ENCRYPT ApsaraDB for HBase ApsaraDB for HBase main function: • Network Isolation and white list; • HAS authentication; • …
  • 22. hosted byIntroduction to Apache HBase Security and Security of ApsaraDB for HBase text • Instal MIT kerberos at the locality; • Set up local krb5.conf,the Kerberos service address,realm and so on of the HBase; • User name and local system user name should be the same, If got no user, need create; • Set up security configuration in hbase-site.xml,core-site.xml,hdfs-site.xml.For example:hadoop.security.authentication,dfs.namenode.kerberos.principal; • Do kinit, run kinit throw user passwor mode or keytab, get the legal user tiket; • Access to hbase throw hbase shell . VS • Set up hbase zookeeper address, user’s password and name; • Access to hbase throw hbase shell . Introduction to ApsaraDB for HBase Autnentication
  • 23. hosted byIntroduction to Apache HBase Security and Security of ApsaraDB for HBase Security network isolation/whitelist/Authentication Cost Reduce the cost of operation and maintenance, perfermance Affinity Simplified configuration;easy to use Userexperience User’s needs start basically from security then low cost then good experience User experience
  • 24. hosted by ApsaraDB for HBase Optimization base on HAS2.2
  • 25. hosted by ApsaraDB for HBase Optimization base on HAS Basical Introduction Why choice HAS for us? • Kerberos is the only means to enable hadoop security • Compatible with existting security mechanism :ACL、 keberos … • Easy to deploy • Easy to use for client • Good scalability • Performance is ok • Low operating cost; • …
  • 26. hosted by ApsaraDB for HBase Optimization base on HAS High availability backend Impement zookeeper like backend for storage of user name/password/whitelist host/kdcconf Account password management Using plugin for password and account management Security and practice Whitelist for hosts and other securities ApsaraDB for HBase’s authentication improvement
  • 27. hosted by ApsaraDB for HBase Optimization base on HAS Security and practice • HA for HAS server • White list for host access • Plugins use HTTPS(Initialiazation/usage) • Configurable salting algorithm • Backward compatible kerberos and all Hbase security • Ops tools : one button to deploy
  • 28. hosted by ApsaraDB for HBase Optimization base on HAS Account password management Aliyun Client/Server plugin mode • Client can initialize with their user/password/hosts • Client pass user/password throw https • Server plugin verify from storage with salt • The entire process is safe • Configure free,easy to use; Other plugin mode: We have done some service on has like:
  • 29. hosted by High availability backend Mysql not fit for cloud environment Single mysql is not ha , 2 or 3 node ha mysql not fit for cloud environment deployment , k-v format, zk is enough At least 3 replicas ,more available than mysql Most value is less than 50B , less memory than mysql and enough performance ApsaraDB for HBase Optimization base on HAS High availability for backend storage Less resource and enough performance
  • 30. hosted by ApsaraDB for HBase Optimization base on HAS Possible user case User consulation case: • All users with same export Ip • ACL is not suitable • Users need different authentication On Cloud almost one small cluster for on user; But private cloud and big cluster is different;
  • 31. hosted by Outlook and Summary2.3
  • 32. hosted by Outlook and Summary • For the security of ApsaraDB for HBase, we got network ioslation 、whitelist and other mechanism . As authentication ,we adopt HAS. • We have do some optimization for HAS such as ha、customized plugins. • The secure ApsaraDB for HBase is ready now. https://www.aliyun.com/product/hbase?spm=5176.8142029.388261.280.702b614asnEokD