SlideShare a Scribd company logo
hosted by
HBase at Xiaomi
Guanghao Zhang
zghao@apache.org
hosted by
About Xiaomi
● Founded in 2010
● Worldwide smartphone No.4 shipments, Q2 2018
● 300+ million global users
● Products: smart phone, TV, AI speaker, smart band…
hosted by
Our HBase Team
● 2 PMC members
● 1 Committer
● 3 HBase Contributers
hosted by
Content 01
02
03
Xiaomi HBase
Quota and Throttling
Synchronous Replication
hosted by
Xiaomi HBase01
hosted by
Architecture
HDFS
HBase Yarn
SDS FDS EMQ MR Spark
Zookeeper
OpenTSDB
hosted by
Clusters
IDC
● 5 data centers (China), 30+ online clusters / 3 offline clusters
AWS / Alibaba Cloud / KS Cloud / Azure
● 16 clusters (China/Singapore/US/Europe/India/Russia)
hosted by
Backup
RegionServer
RegionServer
RegionServer
WAL
Active cluster
RegionServer
RegionServer
Standby cluster
replication
Snapshot
Object Storage (S3)
Message Queue
replication
Hot Backup
Cold Backup
Point-in-time
recovery
HDFS
hosted by
Multi-tenancy
Physical isolation
● Independent HDFS cluster for HBase
● Independent HBase cluster for important business
● Online serving cluster: SSD vs HDD
● Offline processing cluster
Quota and throttling
hosted by
Quota and throttling02
hosted by
Rpc Throttling
Channel
Channel
Channel
Listener
Reader
Reader
Reader
Scheduler
Queue
Queue
Queue
Handler
Handler
Handler
Handler
Handler
Handler
Throttling when handler start
to run request
hosted by
Quota Type
User
Table
Namespace
User ⇒ Table
User ⇒ Namespace
Read
Write
Requests number
Requests size
1. Hard limit
2. If any one quota exceed, then
throttling
hosted by
More Quota Types at Xiaomi
RegionServer Quota: hard limit
Request Unit: calculate both number and size
● Read capacity unit: 1KB/sec
● Write capacity unit: 1KB/sec
Soft limit: allow to exceed user’s quota when regionserver quota not exceed
hosted by
Other Improvements
Switch to start/stop throttling
Metrics and UI support
Handle ThrottlingException in client
● DoNotRetryNowException
● Avoid MR/Spark job failed by throttling
Punishment mechanism for huge requests
hosted by
Synchronous Replication03
hosted by
Async Replication
RegionServer
RegionServer
RegionServer
WAL
Active cluster
Client
sync async
RegionServer
RegionServer
RegionServer
Standby cluster
replication
write
read
hosted by
Async Replication
RegionServer
RegionServer
RegionServer
WAL
Active cluster
Client
sync async
RegionServer
RegionServer
RegionServer
Standby cluster
replication
read
inconsistent
data
crashed
crashed
crashed
hosted by
Sync Replication
RegionServer
RegionServer
RegionServer
WAL
Active cluster
Client
sync async
RegionServer
RegionServer
RegionServer
Standby cluster
replication
write
read
Remote WAL
sync
hosted by
Sync Replication
RegionServer
RegionServer
RegionServer
WAL
Active cluster
Client
sync async
RegionServer
RegionServer
RegionServer
Standby cluster
replication
read
Remote WAL
sync
crashed
crashed
crashed
replay
hosted by
Sync Replication State
Active
Downgrade Active Standby
hosted by
Setup Sync Replication
Active
Downgrade Active
StandbyDowngrade Active
source cluster peer cluster
Downgrade Active
Standby
replication
Step 1
Step 2
Step 3
1. Client read/write active cluster
2. Standby cluster accept async
replication request
replication
replication
hosted by
DualAsyncFSWAL
Add a remoteWALDir to ReplicationPeerConfig
Concurrent write WAL and remote WAL
Write WAL
Write remote
WAL
Client
Success Success Success Same data
Success Fail/Timeout Timeout
Source cluster may has
more data
Fail/Timeout Success Timeout
Peer cluster may has more
data
Fail Fail Fail Same data
Timeout Timeout Timeout
Source or peer cluster may
has more data
hosted by
One RegionServer Crashed
Active
Standby
source cluster peer cluster
Standby
replication
Step 1
Step 2
Step 3
When one regionserver crashed, it
need copy WALs to remote WAL
dir first. Then failover with normal
way.
replication
replication
Active
Active
StandbyRegionServercrashed
RegionServercrashed
WAL
Failover
hosted by
One RegionServer Crashed
Active
source cluster peer cluster
StandbyStep 3
replication
Failover
Case 1: source cluster has more data
Solution: It will replicate data to peer cluster,
so the data will eventually consistent.
Case 2: peer cluster has more data
Solution: Active cluster will copy WAL (which
need to split) to remote WAL dir first, and the
original remote WAL will be replaced. So the
data will eventually consistent.
hosted by
Standby Cluster Crashed
Active
StandbyDowngrade Active
source cluster peer cluster
Standby
replication
Step 1
Step 2
Step 3
The async replication will stuck
when standby crashed. The block
data will be replicated to standby
cluster when it come back.
replication
replication
Active Standbycrashed
crashed
hosted by
Standby Cluster Crashed
Active
source cluster peer cluster
StandbyStep 3
replication
Case 1: source cluster has more data
Solution: It will replicate data to peer cluster,
so the data will eventually consistent.
Case 2: peer cluster has more data
Solution: The remote WALs not used
anymore and will be deleted when the async
replication replicate the original WALs to peer
cluster. So the data will eventually consistent.
hosted by
Active Cluster Crashed
source cluster peer cluster
replication
Step 1
Step 2
Step 3
When source cluster come back
and transit it to standby, it will skip
to replay the original WALs.
replication
replication
Active
Active
crashed
crashed
Standby
Downgrade Active
ActiveStandby
1. Rename remote WAL dir
2. Replay remote WALs
3. Client read/write peer cluster
4. peer cluster doesn’t accept
replication request
hosted by
Active Cluster Crashed
source cluster peer cluster
Step 3
replication
ActiveStandby
Case 1: source cluster has more data
Solution: It will skip to replay the original
WALs(which has been replayed in peer
cluster) and accept replication request which
from peer cluster. So the data will eventually
consistent.
Case 2: peer cluster has more data
Solution: It will replay the remote WALs and
replicate it to source cluster. So the data will
eventually consistent.
hosted by
Sync Replication State Constraint
Active Downgrade Active Standby
Write remote WAL Yes No No
Accept client’s read/write request Yes Yes No
Accept async replication request No No Yes
hosted by
Replication: Async vs Sync
Async Replication Sync Replication
Read Path No affect No affect
Write Path No affect Write extra remote WAL
Network bandwidth 100% for async
replication.
100% for async + 100% for remote
WAL.
Storage space No affect Need extra space for remote WAL
Eventual
Consistency
No if active cluster
crashed
Yes
Availability Unavailable when master
crash
Few time for waiting replay remote
log.
Operational
Complexity
Simple More complex, Need to transition
cluster state by hand or your
services.
hosted by
Performance
YCSB 10^8 operations, 100% Put
Qps: Almost 14% decline compared with async replication
regionserver.heapsize=8g, datanode.heapsize=4g, CMS-gc, disk=1*4T(HDD), cpu=24core
Qps Avg latency P99 latency P999 latency
Async replication 43K 3ms <49.6ms <567ms
Sync replication 37K 3.5ms <74ms <558ms
hosted by
Performance
hosted by
Sync Replication
HBASE-19064: The idea comes from Alibaba's presentation on HBaseCon Asia 2017
HBASE-20422: Synchronous replication for HBase phase II (OPEN)
Sync Replication Xiaomi solution Alibaba solution
Base branch master 0.94
Open source will be released in hbase 3.0 internal solution
Qps 14% decline compared with async replication 2% decline compared with async replication
hosted by
Thanks
Guanghao Zhang
zghao@apache.org

More Related Content

What's hot

HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...
HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...
HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...
Michael Stack
 
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life InsuranceHBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
Michael Stack
 
HBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
HBaseCon 2012 | Building a Large Search Platform on a Shoestring BudgetHBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
HBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
Cloudera, Inc.
 
HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...
HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...
HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...
Michael Stack
 
Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)
DataWorks Summit
 
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsightOptimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
HBaseCon
 
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
Cloudera, Inc.
 
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBase Read High Availability Using Timeline-Consistent Region ReplicasHBase Read High Availability Using Timeline-Consistent Region Replicas
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBaseCon
 
HBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and Cloud
HBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and CloudHBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and Cloud
HBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and Cloud
Michael Stack
 
HBaseConAsia2018 Keynote1: Apache HBase Project Status
HBaseConAsia2018 Keynote1: Apache HBase Project StatusHBaseConAsia2018 Keynote1: Apache HBase Project Status
HBaseConAsia2018 Keynote1: Apache HBase Project Status
Michael Stack
 
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
Cloudera, Inc.
 
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
HBaseCon
 
Tales from the Cloudera Field
Tales from the Cloudera FieldTales from the Cloudera Field
Tales from the Cloudera Field
HBaseCon
 
Scaling HDFS at Xiaomi
Scaling HDFS at XiaomiScaling HDFS at Xiaomi
Scaling HDFS at Xiaomi
DataWorks Summit
 
Supporting Apache HBase : Troubleshooting and Supportability Improvements
Supporting Apache HBase : Troubleshooting and Supportability ImprovementsSupporting Apache HBase : Troubleshooting and Supportability Improvements
Supporting Apache HBase : Troubleshooting and Supportability Improvements
DataWorks Summit
 
HBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsight
HBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsightHBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsight
HBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsight
HBaseCon
 
HBase Tales From the Trenches - Short stories about most common HBase operati...
HBase Tales From the Trenches - Short stories about most common HBase operati...HBase Tales From the Trenches - Short stories about most common HBase operati...
HBase Tales From the Trenches - Short stories about most common HBase operati...
DataWorks Summit
 
Practical NoSQL: Accumulo's dirlist Example
Practical NoSQL: Accumulo's dirlist ExamplePractical NoSQL: Accumulo's dirlist Example
Practical NoSQL: Accumulo's dirlist Example
DataWorks Summit
 
HBaseCon 2015: HBase at Scale in an Online and High-Demand Environment
HBaseCon 2015: HBase at Scale in an Online and  High-Demand EnvironmentHBaseCon 2015: HBase at Scale in an Online and  High-Demand Environment
HBaseCon 2015: HBase at Scale in an Online and High-Demand Environment
HBaseCon
 
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
HBaseCon
 

What's hot (20)

HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...
HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...
HBaseConAsia2018 Track2-1: Kerberos-based Big Data Security Solution and Prac...
 
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life InsuranceHBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
 
HBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
HBaseCon 2012 | Building a Large Search Platform on a Shoestring BudgetHBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
HBaseCon 2012 | Building a Large Search Platform on a Shoestring Budget
 
HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...
HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...
HBaseConAsia2018 Track2-3: Bringing MySQL Compatibility to HBase using Databa...
 
Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)
 
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsightOptimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
 
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
 
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBase Read High Availability Using Timeline-Consistent Region ReplicasHBase Read High Availability Using Timeline-Consistent Region Replicas
HBase Read High Availability Using Timeline-Consistent Region Replicas
 
HBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and Cloud
HBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and CloudHBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and Cloud
HBaseConAsia2018 Keynote 2: Recent Development of HBase in Alibaba and Cloud
 
HBaseConAsia2018 Keynote1: Apache HBase Project Status
HBaseConAsia2018 Keynote1: Apache HBase Project StatusHBaseConAsia2018 Keynote1: Apache HBase Project Status
HBaseConAsia2018 Keynote1: Apache HBase Project Status
 
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster
 
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
 
Tales from the Cloudera Field
Tales from the Cloudera FieldTales from the Cloudera Field
Tales from the Cloudera Field
 
Scaling HDFS at Xiaomi
Scaling HDFS at XiaomiScaling HDFS at Xiaomi
Scaling HDFS at Xiaomi
 
Supporting Apache HBase : Troubleshooting and Supportability Improvements
Supporting Apache HBase : Troubleshooting and Supportability ImprovementsSupporting Apache HBase : Troubleshooting and Supportability Improvements
Supporting Apache HBase : Troubleshooting and Supportability Improvements
 
HBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsight
HBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsightHBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsight
HBaseCon 2015: Optimizing HBase for the Cloud in Microsoft Azure HDInsight
 
HBase Tales From the Trenches - Short stories about most common HBase operati...
HBase Tales From the Trenches - Short stories about most common HBase operati...HBase Tales From the Trenches - Short stories about most common HBase operati...
HBase Tales From the Trenches - Short stories about most common HBase operati...
 
Practical NoSQL: Accumulo's dirlist Example
Practical NoSQL: Accumulo's dirlist ExamplePractical NoSQL: Accumulo's dirlist Example
Practical NoSQL: Accumulo's dirlist Example
 
HBaseCon 2015: HBase at Scale in an Online and High-Demand Environment
HBaseCon 2015: HBase at Scale in an Online and  High-Demand EnvironmentHBaseCon 2015: HBase at Scale in an Online and  High-Demand Environment
HBaseCon 2015: HBase at Scale in an Online and High-Demand Environment
 
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
 

Similar to HBaseConAsia2018 Track1-3: HBase at Xiaomi

Multi site Clustering with Windows Server 2008 Enterprise
Multi site Clustering with Windows Server 2008 EnterpriseMulti site Clustering with Windows Server 2008 Enterprise
Multi site Clustering with Windows Server 2008 Enterprise
Paulo Freitas
 
Data center disaster recovery.ppt
Data center disaster recovery.ppt Data center disaster recovery.ppt
Data center disaster recovery.ppt
omalreda
 
MNPHP Scalable Architecture 101 - Feb 3 2011
MNPHP Scalable Architecture 101 - Feb 3 2011MNPHP Scalable Architecture 101 - Feb 3 2011
MNPHP Scalable Architecture 101 - Feb 3 2011
Mike Willbanks
 
Apache ignite as in-memory computing platform
Apache ignite as in-memory computing platformApache ignite as in-memory computing platform
Apache ignite as in-memory computing platform
Surinder Mehra
 
VMworld Europe 2014: Virtual SAN Best Practices and Use Cases
VMworld Europe 2014: Virtual SAN Best Practices and Use CasesVMworld Europe 2014: Virtual SAN Best Practices and Use Cases
VMworld Europe 2014: Virtual SAN Best Practices and Use Cases
VMworld
 
Hyper v r2 deep dive
Hyper v r2 deep diveHyper v r2 deep dive
Hyper v r2 deep dive
Concentrated Technology
 
Plandas-CacheCloud
Plandas-CacheCloudPlandas-CacheCloud
Plandas-CacheCloud
Gyuman Cho
 
Serverless (Distributed computing)
Serverless (Distributed computing)Serverless (Distributed computing)
Serverless (Distributed computing)
Sri Prasanna
 
Spinnaker VLDB 2011
Spinnaker VLDB 2011Spinnaker VLDB 2011
Spinnaker VLDB 2011
sandeep_tata
 
HA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybridHA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybrid
James Serra
 
AlwaysON Basics
AlwaysON BasicsAlwaysON Basics
AlwaysON Basics
Harsh Chawla
 
Implementing dr w. hyper v clustering
Implementing dr w. hyper v clusteringImplementing dr w. hyper v clustering
Implementing dr w. hyper v clustering
Concentrated Technology
 
MySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveMySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspective
Ulf Wendel
 
Designing High Availability for HashiCorp Vault in AWS
Designing High Availability for HashiCorp Vault in AWSDesigning High Availability for HashiCorp Vault in AWS
Designing High Availability for HashiCorp Vault in AWS
☁ Bryan Krausen
 
Hadoop 3.0 - Revolution or evolution?
Hadoop 3.0 - Revolution or evolution?Hadoop 3.0 - Revolution or evolution?
Hadoop 3.0 - Revolution or evolution?
Uwe Printz
 
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Виталий Стародубцев
 
07_DP_300T00A_HA_Disaster_Recovery.pptx
07_DP_300T00A_HA_Disaster_Recovery.pptx07_DP_300T00A_HA_Disaster_Recovery.pptx
07_DP_300T00A_HA_Disaster_Recovery.pptx
KareemBullard1
 
Sql saturday dc vm ware
Sql saturday dc vm wareSql saturday dc vm ware
Sql saturday dc vm ware
Joseph D'Antoni
 
Invalidation-Based Protocols for Replicated Datastores
Invalidation-Based Protocols for Replicated DatastoresInvalidation-Based Protocols for Replicated Datastores
Invalidation-Based Protocols for Replicated Datastores
Antonios Katsarakis
 
Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...
Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...
Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...
Cloudera, Inc.
 

Similar to HBaseConAsia2018 Track1-3: HBase at Xiaomi (20)

Multi site Clustering with Windows Server 2008 Enterprise
Multi site Clustering with Windows Server 2008 EnterpriseMulti site Clustering with Windows Server 2008 Enterprise
Multi site Clustering with Windows Server 2008 Enterprise
 
Data center disaster recovery.ppt
Data center disaster recovery.ppt Data center disaster recovery.ppt
Data center disaster recovery.ppt
 
MNPHP Scalable Architecture 101 - Feb 3 2011
MNPHP Scalable Architecture 101 - Feb 3 2011MNPHP Scalable Architecture 101 - Feb 3 2011
MNPHP Scalable Architecture 101 - Feb 3 2011
 
Apache ignite as in-memory computing platform
Apache ignite as in-memory computing platformApache ignite as in-memory computing platform
Apache ignite as in-memory computing platform
 
VMworld Europe 2014: Virtual SAN Best Practices and Use Cases
VMworld Europe 2014: Virtual SAN Best Practices and Use CasesVMworld Europe 2014: Virtual SAN Best Practices and Use Cases
VMworld Europe 2014: Virtual SAN Best Practices and Use Cases
 
Hyper v r2 deep dive
Hyper v r2 deep diveHyper v r2 deep dive
Hyper v r2 deep dive
 
Plandas-CacheCloud
Plandas-CacheCloudPlandas-CacheCloud
Plandas-CacheCloud
 
Serverless (Distributed computing)
Serverless (Distributed computing)Serverless (Distributed computing)
Serverless (Distributed computing)
 
Spinnaker VLDB 2011
Spinnaker VLDB 2011Spinnaker VLDB 2011
Spinnaker VLDB 2011
 
HA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybridHA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybrid
 
AlwaysON Basics
AlwaysON BasicsAlwaysON Basics
AlwaysON Basics
 
Implementing dr w. hyper v clustering
Implementing dr w. hyper v clusteringImplementing dr w. hyper v clustering
Implementing dr w. hyper v clustering
 
MySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveMySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspective
 
Designing High Availability for HashiCorp Vault in AWS
Designing High Availability for HashiCorp Vault in AWSDesigning High Availability for HashiCorp Vault in AWS
Designing High Availability for HashiCorp Vault in AWS
 
Hadoop 3.0 - Revolution or evolution?
Hadoop 3.0 - Revolution or evolution?Hadoop 3.0 - Revolution or evolution?
Hadoop 3.0 - Revolution or evolution?
 
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
 
07_DP_300T00A_HA_Disaster_Recovery.pptx
07_DP_300T00A_HA_Disaster_Recovery.pptx07_DP_300T00A_HA_Disaster_Recovery.pptx
07_DP_300T00A_HA_Disaster_Recovery.pptx
 
Sql saturday dc vm ware
Sql saturday dc vm wareSql saturday dc vm ware
Sql saturday dc vm ware
 
Invalidation-Based Protocols for Replicated Datastores
Invalidation-Based Protocols for Replicated DatastoresInvalidation-Based Protocols for Replicated Datastores
Invalidation-Based Protocols for Replicated Datastores
 
Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...
Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...
Hadoop World 2011: HDFS Name Node High Availablity - Aaron Myers, Cloudera & ...
 

More from Michael Stack

hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloudhbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
Michael Stack
 
hbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinteresthbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinterest
Michael Stack
 
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltdhbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
Michael Stack
 
hbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didihbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didi
Michael Stack
 
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
Michael Stack
 
hbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencenthbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencent
Michael Stack
 
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
Michael Stack
 
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
Michael Stack
 
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Componenthbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
Michael Stack
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
Michael Stack
 
hbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomihbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomi
Michael Stack
 
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Sparkhbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
Michael Stack
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
Michael Stack
 
hbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solutionhbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solution
Michael Stack
 
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memoryhbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
Michael Stack
 
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACLhbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
Michael Stack
 
hbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBasehbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBase
Michael Stack
 
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
Michael Stack
 
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
Michael Stack
 
HBaseConAsia2019 Keynote
HBaseConAsia2019 KeynoteHBaseConAsia2019 Keynote
HBaseConAsia2019 Keynote
Michael Stack
 

More from Michael Stack (20)

hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloudhbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
 
hbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinteresthbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinterest
 
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltdhbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd
 
hbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didihbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didi
 
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
 
hbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencenthbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencent
 
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
 
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
 
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Componenthbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
 
hbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomihbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomi
 
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Sparkhbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
 
hbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solutionhbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solution
 
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memoryhbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
 
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACLhbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
 
hbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBasehbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBase
 
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
 
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
 
HBaseConAsia2019 Keynote
HBaseConAsia2019 KeynoteHBaseConAsia2019 Keynote
HBaseConAsia2019 Keynote
 

Recently uploaded

Rent remote desktop server mangohost .net
Rent remote desktop server mangohost .netRent remote desktop server mangohost .net
Rent remote desktop server mangohost .net
pdfsubmission50
 
Portugal Dreamin 24 - How to easily use an API with Flows
Portugal Dreamin 24  - How to easily use an API with FlowsPortugal Dreamin 24  - How to easily use an API with Flows
Portugal Dreamin 24 - How to easily use an API with Flows
Thierry TROUIN ☁
 
Why Your Business Needs a Professional Web Design Company UAE
Why Your Business Needs a Professional Web Design Company UAEWhy Your Business Needs a Professional Web Design Company UAE
Why Your Business Needs a Professional Web Design Company UAE
adelewhite125
 
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docxBitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
SFC Today
 
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
mahigarg2024#G05
 
Effective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptxEffective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptx
AirtoryInc
 
UMN degree offer diploma Transcript
UMN degree offer diploma TranscriptUMN degree offer diploma Transcript
UMN degree offer diploma Transcript
cenocb
 
Software Defined Networking, Concepts and Practical Implementations
Software Defined Networking, Concepts and Practical ImplementationsSoftware Defined Networking, Concepts and Practical Implementations
Software Defined Networking, Concepts and Practical Implementations
Bangladesh Network Operators Group
 
Tarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy FearsTarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur
 
How Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital TransformationHow Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital Transformation
Sweet Potato Tec
 
AWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaipromAWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaiprom
ธนาพัฒน์ ลิ้มสายพรหม
 
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
elbertablack
 
DASH, presented by Elly Tawhai at PacNOG 33
DASH, presented by Elly Tawhai at PacNOG 33DASH, presented by Elly Tawhai at PacNOG 33
DASH, presented by Elly Tawhai at PacNOG 33
APNIC
 
Build a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat LimsaipromBuild a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat Limsaiprom
TanapatLimsaiprom1
 
2023. Archive - Gigabajtos selfpublisher homepage
2023. Archive - Gigabajtos selfpublisher homepage2023. Archive - Gigabajtos selfpublisher homepage
2023. Archive - Gigabajtos selfpublisher homepage
Zsolt Nemeth
 
Girls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in City
dilbaagsingh0898
 
Web development Platform Constraints.pptx
Web development Platform Constraints.pptxWeb development Platform Constraints.pptx
Web development Platform Constraints.pptx
ssuser2f6682
 
Career Development Advice for Network Engineers across the Pacific, presented...
Career Development Advice for Network Engineers across the Pacific, presented...Career Development Advice for Network Engineers across the Pacific, presented...
Career Development Advice for Network Engineers across the Pacific, presented...
APNIC
 
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptxIot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
DeepakKumar862274
 
Ontology for the semantic enhancement, database definition and management and...
Ontology for the semantic enhancement, database definition and management and...Ontology for the semantic enhancement, database definition and management and...
Ontology for the semantic enhancement, database definition and management and...
Edward Blurock
 

Recently uploaded (20)

Rent remote desktop server mangohost .net
Rent remote desktop server mangohost .netRent remote desktop server mangohost .net
Rent remote desktop server mangohost .net
 
Portugal Dreamin 24 - How to easily use an API with Flows
Portugal Dreamin 24  - How to easily use an API with FlowsPortugal Dreamin 24  - How to easily use an API with Flows
Portugal Dreamin 24 - How to easily use an API with Flows
 
Why Your Business Needs a Professional Web Design Company UAE
Why Your Business Needs a Professional Web Design Company UAEWhy Your Business Needs a Professional Web Design Company UAE
Why Your Business Needs a Professional Web Design Company UAE
 
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docxBitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
 
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
 
Effective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptxEffective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptx
 
UMN degree offer diploma Transcript
UMN degree offer diploma TranscriptUMN degree offer diploma Transcript
UMN degree offer diploma Transcript
 
Software Defined Networking, Concepts and Practical Implementations
Software Defined Networking, Concepts and Practical ImplementationsSoftware Defined Networking, Concepts and Practical Implementations
Software Defined Networking, Concepts and Practical Implementations
 
Tarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy FearsTarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy Fears
 
How Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital TransformationHow Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital Transformation
 
AWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaipromAWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaiprom
 
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
 
DASH, presented by Elly Tawhai at PacNOG 33
DASH, presented by Elly Tawhai at PacNOG 33DASH, presented by Elly Tawhai at PacNOG 33
DASH, presented by Elly Tawhai at PacNOG 33
 
Build a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat LimsaipromBuild a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat Limsaiprom
 
2023. Archive - Gigabajtos selfpublisher homepage
2023. Archive - Gigabajtos selfpublisher homepage2023. Archive - Gigabajtos selfpublisher homepage
2023. Archive - Gigabajtos selfpublisher homepage
 
Girls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Shimla 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Web development Platform Constraints.pptx
Web development Platform Constraints.pptxWeb development Platform Constraints.pptx
Web development Platform Constraints.pptx
 
Career Development Advice for Network Engineers across the Pacific, presented...
Career Development Advice for Network Engineers across the Pacific, presented...Career Development Advice for Network Engineers across the Pacific, presented...
Career Development Advice for Network Engineers across the Pacific, presented...
 
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptxIot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
 
Ontology for the semantic enhancement, database definition and management and...
Ontology for the semantic enhancement, database definition and management and...Ontology for the semantic enhancement, database definition and management and...
Ontology for the semantic enhancement, database definition and management and...
 

HBaseConAsia2018 Track1-3: HBase at Xiaomi

  • 1. hosted by HBase at Xiaomi Guanghao Zhang zghao@apache.org
  • 2. hosted by About Xiaomi ● Founded in 2010 ● Worldwide smartphone No.4 shipments, Q2 2018 ● 300+ million global users ● Products: smart phone, TV, AI speaker, smart band…
  • 3. hosted by Our HBase Team ● 2 PMC members ● 1 Committer ● 3 HBase Contributers
  • 4. hosted by Content 01 02 03 Xiaomi HBase Quota and Throttling Synchronous Replication
  • 6. hosted by Architecture HDFS HBase Yarn SDS FDS EMQ MR Spark Zookeeper OpenTSDB
  • 7. hosted by Clusters IDC ● 5 data centers (China), 30+ online clusters / 3 offline clusters AWS / Alibaba Cloud / KS Cloud / Azure ● 16 clusters (China/Singapore/US/Europe/India/Russia)
  • 8. hosted by Backup RegionServer RegionServer RegionServer WAL Active cluster RegionServer RegionServer Standby cluster replication Snapshot Object Storage (S3) Message Queue replication Hot Backup Cold Backup Point-in-time recovery HDFS
  • 9. hosted by Multi-tenancy Physical isolation ● Independent HDFS cluster for HBase ● Independent HBase cluster for important business ● Online serving cluster: SSD vs HDD ● Offline processing cluster Quota and throttling
  • 10. hosted by Quota and throttling02
  • 12. hosted by Quota Type User Table Namespace User ⇒ Table User ⇒ Namespace Read Write Requests number Requests size 1. Hard limit 2. If any one quota exceed, then throttling
  • 13. hosted by More Quota Types at Xiaomi RegionServer Quota: hard limit Request Unit: calculate both number and size ● Read capacity unit: 1KB/sec ● Write capacity unit: 1KB/sec Soft limit: allow to exceed user’s quota when regionserver quota not exceed
  • 14. hosted by Other Improvements Switch to start/stop throttling Metrics and UI support Handle ThrottlingException in client ● DoNotRetryNowException ● Avoid MR/Spark job failed by throttling Punishment mechanism for huge requests
  • 16. hosted by Async Replication RegionServer RegionServer RegionServer WAL Active cluster Client sync async RegionServer RegionServer RegionServer Standby cluster replication write read
  • 17. hosted by Async Replication RegionServer RegionServer RegionServer WAL Active cluster Client sync async RegionServer RegionServer RegionServer Standby cluster replication read inconsistent data crashed crashed crashed
  • 18. hosted by Sync Replication RegionServer RegionServer RegionServer WAL Active cluster Client sync async RegionServer RegionServer RegionServer Standby cluster replication write read Remote WAL sync
  • 19. hosted by Sync Replication RegionServer RegionServer RegionServer WAL Active cluster Client sync async RegionServer RegionServer RegionServer Standby cluster replication read Remote WAL sync crashed crashed crashed replay
  • 20. hosted by Sync Replication State Active Downgrade Active Standby
  • 21. hosted by Setup Sync Replication Active Downgrade Active StandbyDowngrade Active source cluster peer cluster Downgrade Active Standby replication Step 1 Step 2 Step 3 1. Client read/write active cluster 2. Standby cluster accept async replication request replication replication
  • 22. hosted by DualAsyncFSWAL Add a remoteWALDir to ReplicationPeerConfig Concurrent write WAL and remote WAL Write WAL Write remote WAL Client Success Success Success Same data Success Fail/Timeout Timeout Source cluster may has more data Fail/Timeout Success Timeout Peer cluster may has more data Fail Fail Fail Same data Timeout Timeout Timeout Source or peer cluster may has more data
  • 23. hosted by One RegionServer Crashed Active Standby source cluster peer cluster Standby replication Step 1 Step 2 Step 3 When one regionserver crashed, it need copy WALs to remote WAL dir first. Then failover with normal way. replication replication Active Active StandbyRegionServercrashed RegionServercrashed WAL Failover
  • 24. hosted by One RegionServer Crashed Active source cluster peer cluster StandbyStep 3 replication Failover Case 1: source cluster has more data Solution: It will replicate data to peer cluster, so the data will eventually consistent. Case 2: peer cluster has more data Solution: Active cluster will copy WAL (which need to split) to remote WAL dir first, and the original remote WAL will be replaced. So the data will eventually consistent.
  • 25. hosted by Standby Cluster Crashed Active StandbyDowngrade Active source cluster peer cluster Standby replication Step 1 Step 2 Step 3 The async replication will stuck when standby crashed. The block data will be replicated to standby cluster when it come back. replication replication Active Standbycrashed crashed
  • 26. hosted by Standby Cluster Crashed Active source cluster peer cluster StandbyStep 3 replication Case 1: source cluster has more data Solution: It will replicate data to peer cluster, so the data will eventually consistent. Case 2: peer cluster has more data Solution: The remote WALs not used anymore and will be deleted when the async replication replicate the original WALs to peer cluster. So the data will eventually consistent.
  • 27. hosted by Active Cluster Crashed source cluster peer cluster replication Step 1 Step 2 Step 3 When source cluster come back and transit it to standby, it will skip to replay the original WALs. replication replication Active Active crashed crashed Standby Downgrade Active ActiveStandby 1. Rename remote WAL dir 2. Replay remote WALs 3. Client read/write peer cluster 4. peer cluster doesn’t accept replication request
  • 28. hosted by Active Cluster Crashed source cluster peer cluster Step 3 replication ActiveStandby Case 1: source cluster has more data Solution: It will skip to replay the original WALs(which has been replayed in peer cluster) and accept replication request which from peer cluster. So the data will eventually consistent. Case 2: peer cluster has more data Solution: It will replay the remote WALs and replicate it to source cluster. So the data will eventually consistent.
  • 29. hosted by Sync Replication State Constraint Active Downgrade Active Standby Write remote WAL Yes No No Accept client’s read/write request Yes Yes No Accept async replication request No No Yes
  • 30. hosted by Replication: Async vs Sync Async Replication Sync Replication Read Path No affect No affect Write Path No affect Write extra remote WAL Network bandwidth 100% for async replication. 100% for async + 100% for remote WAL. Storage space No affect Need extra space for remote WAL Eventual Consistency No if active cluster crashed Yes Availability Unavailable when master crash Few time for waiting replay remote log. Operational Complexity Simple More complex, Need to transition cluster state by hand or your services.
  • 31. hosted by Performance YCSB 10^8 operations, 100% Put Qps: Almost 14% decline compared with async replication regionserver.heapsize=8g, datanode.heapsize=4g, CMS-gc, disk=1*4T(HDD), cpu=24core Qps Avg latency P99 latency P999 latency Async replication 43K 3ms <49.6ms <567ms Sync replication 37K 3.5ms <74ms <558ms
  • 33. hosted by Sync Replication HBASE-19064: The idea comes from Alibaba's presentation on HBaseCon Asia 2017 HBASE-20422: Synchronous replication for HBase phase II (OPEN) Sync Replication Xiaomi solution Alibaba solution Base branch master 0.94 Open source will be released in hbase 3.0 internal solution Qps 14% decline compared with async replication 2% decline compared with async replication