SlideShare a Scribd company logo
1 of 30
Download to read offline
Phoenix Practice in China Life
Insurance Co., Ltd.
Leo Yuan
yuanliou2015@e-chinalife.com
1 Scenarios
2 Designs
3 Optimizations
4 Problems
5 Future Work
Agenda
Scenarios - Overview
Main Processing
Cluster A
Application Procession
Cluster C
Readonly cluster B
Sync
basic
data
Sync result
Real Time Query
Cluster E
Scenarios - Overview
4 Cluste200+ Nodes(30+ Phoenix Nodes)
Clusters
Data
1300TB+ data
30TB+ biggest table
Querys
Ten million level perday
Processing
Hundreds of MR/Hive/Spark jobs per day
50TB+ Incremental data for update & insert
Scenarios - Overview
step 1 get some money from
company counter
step 2 query operation detail from app
This Is What We Do
Real Cust Rights View System !
Scenarios - Overview
30,000,000 + Incremental data/day 700,000 + users / day
10,000+ records/sec 8,000+ sqls / sec
1 Scenarios
2 Designs
3 Optimizations
4 Problems
5 Future Work
Agenda
Designs - Data Architecture
ü Initialize Data
ü Build Phoenix Index
ü Sync Real Time Data
ü Provide Data Service
China Life Insurance APP
Business
System
SharePlex
Kafka
Spark
Streaming
Cust View System
Phoenix
HBase
Designs - Development Architecture
Data Integration
Data View
Cross-province Integration
Query Distribution Engine
V_constract table
(4+3)
Data Service
Ultimate Real Time Query Service
Data Exchange Initial Data Sync Program
Data Source Business System DataBase
JobSchedule
Monitor
Resourceschedule
monitor
Schedule
Monitor
Real Time Data Link
Incremental Data Sync Program
V_pay table
(4+3)
V_cust service table
(4+3)
V_claim table
(4+3) …
dim
Resourceaccess
control
UserAccess
Control
Privilege
Control
Designs - Physical Architecture
Phoenix Cluster
Configuration DB Weblogic Server Weblogic Server Weblogic Server
F5APP Server
Phoenix Gateway
Kafka
Designs - Real Time Data Sync
cid c_no type amount branch syssource updtime incr_flag commit_time ……
001 001 M 100 000000 V6 2019-06-11 16:37.322 1 2019-06-11 16:37.322 ……
02 002 R 100 000001 V7 2019-06-11 16:38.689 2 2019-06-11 16:38.689 ……
Contract(BeiJing)
cid c_no type amount
01 001 M 100
1、Partition By Global Primary Key
2、Shield Upstream System Table Structure Adjustment
3、No Effect on Normal Stream Process when Data
Supplement
…
… ……
SparkStreaming
Contract(ShangHai)
cid c_no type amount
02 002 R 100
SparkStreaming_compt
Designs - Ultimate Real Time Query Service
label name label type isHolder logic
paidBonus sum holder sql1,2,3
paidMoenyList list holder sql4,5
paidExpire sum Insured sql6
… … … …
label.properties
Analyze
Parameters
Package Sqls
Execue Sqls
Collect Results
Chinalife Insurance APP
Ultimate Real Time Query Service
1 Scenarios
2 Designs
3 Optimizations
4 Problems
5 Future Work
Agenda
Optimizations - Sql Execution Process
DriverManager.getConnection("phoenixUrl")
con.prepareStatement(sql)
pstat.executeQuery()
rs.next()
1、SYSTEM.CATALOG
2、SYSTEM.STATS
3、SYSTEM.LOG
4、 SYSTEM.SEQUENCE
1、Query meta data of table/index
from phoenix server(SYSTEM.CATALOG)
2、Determine the table/index sql need to scan
3、Query statistics information of table/index
from phoenix server(SYSTEM.STATS)
4、Generate scans based on statistics information、meta data、sql
1、Parallelity decided by phoenix.query.threadPoolSize
Optimizations – Phoenix System Table
SYSTEM.CATALOG
SYSTEM.STATS
Describe table/index meta information,
such as
l TABLE NEME
l COLUMN NAME
l SALT_BUCKETS
l UPDATE_CACHE_FREQUENCY
l GUIDE_POST_WIDTH
Describe table/index accurate
statistics information, such as
l GUIDE_POST_KEY
l GUIDE_POSTS_WIDTH
PS:UPDATE STATISTICS TABLE_NEME
Optimizations - RS Group
Hmaster
SYSTEM:CATALOG
SYSTEM:MUTEX
SYSTEM:STATS
CONTRACT
INCOME
PERSON
RS Group 1 RS Group 2
rs1
SYSTEM:CATALOG SYSTEM:STATS
Hmaster
rs2 rs3 rs4 … rs1 rs3 rs4 …rs2
……
SYSTEM:STATSCONTRACTINCOME
PERSON ……
• Metadata Table Isolation to Decrease Impact on Business Table Query
Optimizations - UPDATE_CACHE_FREQUENCY
• Adjust this parameter to decrease hotspot in SYSTEM.CATALOG
p Decide the query frequency of SYSTEM.CATALOG
p Default value is “Always” and will Cause read/write pressure in SYSTEM.CATALOG
p Can be set per Cluster/Table
1. “phoenix.default.update.cache.frequency”: 86412345
2. create table test.test (a varchar not null primary key,b varchar )
SALT_BUCKETS = 10, UPDATE_CACHE_FREQUENCY=86400000;
Optimizations – Salt & Pre-Split
• Data table use salt_buckets, Index table use pre-split
CREATE TABLE …(
)SALT_BUCKETS = 60
CREATE INDEX …
ON …(
…
) INCLUDE (
…
) ASYNC SALT_BUCKETS
= 0
SPLIT ON (
…
)
1、Create index to
ensure that all of query
process stop at index
table
2、use pre split to reduce
the chunk number when
execute a query
3、use async index to
avoid OOM when
building the index table
1、Use
salted
table to
distribute
data evenly
Phoenix Client
write
Phoenix Client
query
Optimizations - Open Offheap
• Minimal read cost to improve query efficiency
BucketCache Configuration Properties
l hbase.bucketcache.combinedcache.enabled
l hbase.bucketcache.ioengine
l hfile.block.cache.size
l hbase.bucketcache.size
l hbase.bucketcache.bucket.sizes
l -XX:MaxDirectMemorySize
Optimizations - Other Configurations
• Region Balance By Table
• G1GC
• Manual MajorCompaction
1 Scenarios
2 Designs
3 Optimizations
4 Problems
5 Future Work
Agenda
Problems - Current time function cause Query performance degradation
• This function will lead to client-cluster interaction frequently
SELECT …
FROM …
WHERE date <=
CURRENT_TIME()
SELECT …
FROM …
WHERE date <=
CURRENT_TIME()
SELECT …
FROM …
WHERE date <= NOW_DATE
replace CURRENT_TIME
In java code to NOW_DATE
①
②
③
Problems - HBase cluster balance abnormal
1.Turn on RSGroup
2.Set two RSGroup default 、my_group
3.Move rs to default and my_group
4.Restart one rs in default
process Problems Resolve
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer
2. balance_rsgroup 'default' abnormal
1. balancer abnormal
Problems - ACL abnormal when come together with RSGroup
1.Turn on RSGroup
2.Turn on ACL
process Problems
1. Non-hbase user can’t creat table
Resolve
1, Use hbase user to create table
2, grant this table 'RWX’ to Non-hbase user
<property>
<name>
hbase.coprocessor.master.classes
</name>
<value>
org.apache.hadoop.hbase.security.
access.AccessController,
org.apache.hadoop.
hbase.rsgroup.RSGroupAdminEndpoint
</value>
</property>
1 Scenarios
2 Designs
3 Optimizations
4 Problems
5 Future Work
Agenda
Future Work - RPC Read/Write Isolation
total queue = hbase.ipc.server.callqueue.handler.factor * handler
read queue = total queue * hbase.ipc.server.callqueue.read.ratio
write queue = total queue * (1- hbase.ipc.server.callqueue.read.ratio )
scan queue = = total queue * hbase.ipc.server.callqueue.read.ratio *
hbase.ipc.server.callqueue.scan.ratio
total queue
write
queue
read queue
scan
queue
Future Work - Compaction Contral
Set Offpeak Time
Set Peak Time Throughput
l hbase.offpeak.end.hour
l hbase.offpeak.start.hour
l hbase.hstore.compaction.throughput.offpeak
l key:hbase.regionserver.throughput.controller
value:org.apache.hadoop.hbase.regionserver.compactions.
PressureAwareCompactionThroughputController
Open Compaction Controller
l hbase.hstore.compaction.throughput.higher.bound
l hbase.hstore.compaction.throughput.lower.bound
Future Work - Join Optimization
SELECT …
FROM ( a1
JOIN (SELECT...
FROM b1
WHERE …) a2
ON a1…. = a2…)
WHERE …
EXPLAIN
Thanks!
yuanliou2015@e-chinalife.com

More Related Content

What's hot

January 2016 Flink Community Update & Roadmap 2016
January 2016 Flink Community Update & Roadmap 2016January 2016 Flink Community Update & Roadmap 2016
January 2016 Flink Community Update & Roadmap 2016Robert Metzger
 
Distributing Queries the Citus Way | PostgresConf US 2018 | Marco Slot
Distributing Queries the Citus Way | PostgresConf US 2018 | Marco SlotDistributing Queries the Citus Way | PostgresConf US 2018 | Marco Slot
Distributing Queries the Citus Way | PostgresConf US 2018 | Marco SlotCitus Data
 
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at RakutenMongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at RakutenMongoDB
 
The Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus StoryThe Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus StoryHanna Kelman
 
ApacheCon: Apache Flink - Fast and Reliable Large-Scale Data Processing
ApacheCon: Apache Flink - Fast and Reliable Large-Scale Data ProcessingApacheCon: Apache Flink - Fast and Reliable Large-Scale Data Processing
ApacheCon: Apache Flink - Fast and Reliable Large-Scale Data ProcessingFabian Hueske
 
Towards sql for streams
Towards sql for streamsTowards sql for streams
Towards sql for streamsRadu Tudoran
 
Real-time Stream Processing with Apache Flink
Real-time Stream Processing with Apache FlinkReal-time Stream Processing with Apache Flink
Real-time Stream Processing with Apache FlinkDataWorks Summit
 
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...Flink Forward
 
Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015
Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015
Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015Robert Metzger
 
Apache Flink@ Strata & Hadoop World London
Apache Flink@ Strata & Hadoop World LondonApache Flink@ Strata & Hadoop World London
Apache Flink@ Strata & Hadoop World LondonStephan Ewen
 
Structured streaming for machine learning
Structured streaming for machine learningStructured streaming for machine learning
Structured streaming for machine learningSeth Hendrickson
 
Taking Spark Streaming to the Next Level with Datasets and DataFrames
Taking Spark Streaming to the Next Level with Datasets and DataFramesTaking Spark Streaming to the Next Level with Datasets and DataFrames
Taking Spark Streaming to the Next Level with Datasets and DataFramesDatabricks
 
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache SparkArbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache SparkDatabricks
 
Apache Beam: A unified model for batch and stream processing data
Apache Beam: A unified model for batch and stream processing dataApache Beam: A unified model for batch and stream processing data
Apache Beam: A unified model for batch and stream processing dataDataWorks Summit/Hadoop Summit
 
Batch and Stream Graph Processing with Apache Flink
Batch and Stream Graph Processing with Apache FlinkBatch and Stream Graph Processing with Apache Flink
Batch and Stream Graph Processing with Apache FlinkVasia Kalavri
 
Data Stream Processing with Apache Flink
Data Stream Processing with Apache FlinkData Stream Processing with Apache Flink
Data Stream Processing with Apache FlinkFabian Hueske
 
Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15
Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15
Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15Vasia Kalavri
 
Flink 0.10 @ Bay Area Meetup (October 2015)
Flink 0.10 @ Bay Area Meetup (October 2015)Flink 0.10 @ Bay Area Meetup (October 2015)
Flink 0.10 @ Bay Area Meetup (October 2015)Stephan Ewen
 
Flink Gelly - Karlsruhe - June 2015
Flink Gelly - Karlsruhe - June 2015Flink Gelly - Karlsruhe - June 2015
Flink Gelly - Karlsruhe - June 2015Andra Lungu
 
Apache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and FriendsApache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and FriendsStephan Ewen
 

What's hot (20)

January 2016 Flink Community Update & Roadmap 2016
January 2016 Flink Community Update & Roadmap 2016January 2016 Flink Community Update & Roadmap 2016
January 2016 Flink Community Update & Roadmap 2016
 
Distributing Queries the Citus Way | PostgresConf US 2018 | Marco Slot
Distributing Queries the Citus Way | PostgresConf US 2018 | Marco SlotDistributing Queries the Citus Way | PostgresConf US 2018 | Marco Slot
Distributing Queries the Citus Way | PostgresConf US 2018 | Marco Slot
 
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at RakutenMongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
 
The Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus StoryThe Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus Story
 
ApacheCon: Apache Flink - Fast and Reliable Large-Scale Data Processing
ApacheCon: Apache Flink - Fast and Reliable Large-Scale Data ProcessingApacheCon: Apache Flink - Fast and Reliable Large-Scale Data Processing
ApacheCon: Apache Flink - Fast and Reliable Large-Scale Data Processing
 
Towards sql for streams
Towards sql for streamsTowards sql for streams
Towards sql for streams
 
Real-time Stream Processing with Apache Flink
Real-time Stream Processing with Apache FlinkReal-time Stream Processing with Apache Flink
Real-time Stream Processing with Apache Flink
 
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
 
Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015
Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015
Architecture of Flink's Streaming Runtime @ ApacheCon EU 2015
 
Apache Flink@ Strata & Hadoop World London
Apache Flink@ Strata & Hadoop World LondonApache Flink@ Strata & Hadoop World London
Apache Flink@ Strata & Hadoop World London
 
Structured streaming for machine learning
Structured streaming for machine learningStructured streaming for machine learning
Structured streaming for machine learning
 
Taking Spark Streaming to the Next Level with Datasets and DataFrames
Taking Spark Streaming to the Next Level with Datasets and DataFramesTaking Spark Streaming to the Next Level with Datasets and DataFrames
Taking Spark Streaming to the Next Level with Datasets and DataFrames
 
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache SparkArbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
 
Apache Beam: A unified model for batch and stream processing data
Apache Beam: A unified model for batch and stream processing dataApache Beam: A unified model for batch and stream processing data
Apache Beam: A unified model for batch and stream processing data
 
Batch and Stream Graph Processing with Apache Flink
Batch and Stream Graph Processing with Apache FlinkBatch and Stream Graph Processing with Apache Flink
Batch and Stream Graph Processing with Apache Flink
 
Data Stream Processing with Apache Flink
Data Stream Processing with Apache FlinkData Stream Processing with Apache Flink
Data Stream Processing with Apache Flink
 
Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15
Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15
Large-scale graph processing with Apache Flink @GraphDevroom FOSDEM'15
 
Flink 0.10 @ Bay Area Meetup (October 2015)
Flink 0.10 @ Bay Area Meetup (October 2015)Flink 0.10 @ Bay Area Meetup (October 2015)
Flink 0.10 @ Bay Area Meetup (October 2015)
 
Flink Gelly - Karlsruhe - June 2015
Flink Gelly - Karlsruhe - June 2015Flink Gelly - Karlsruhe - June 2015
Flink Gelly - Karlsruhe - June 2015
 
Apache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and FriendsApache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and Friends
 

Similar to hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd

Strata+Hadoop 2015 NYC End User Panel on Real-Time Data Analytics
Strata+Hadoop 2015 NYC End User Panel on Real-Time Data AnalyticsStrata+Hadoop 2015 NYC End User Panel on Real-Time Data Analytics
Strata+Hadoop 2015 NYC End User Panel on Real-Time Data AnalyticsSingleStore
 
Flexible and Real-Time Stream Processing with Apache Flink
Flexible and Real-Time Stream Processing with Apache FlinkFlexible and Real-Time Stream Processing with Apache Flink
Flexible and Real-Time Stream Processing with Apache FlinkDataWorks Summit
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Amazon Web Services
 
Scheduling in Linux and Web Servers
Scheduling in Linux and Web ServersScheduling in Linux and Web Servers
Scheduling in Linux and Web ServersDavid Evans
 
Apache Spark 2.0: Faster, Easier, and Smarter
Apache Spark 2.0: Faster, Easier, and SmarterApache Spark 2.0: Faster, Easier, and Smarter
Apache Spark 2.0: Faster, Easier, and SmarterDatabricks
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationYi Pan
 
Streaming data for real time analysis
Streaming data for real time analysisStreaming data for real time analysis
Streaming data for real time analysisAmazon Web Services
 
Yogesh kumar kushwah represent’s
Yogesh kumar kushwah represent’sYogesh kumar kushwah represent’s
Yogesh kumar kushwah represent’sYogesh Kushwah
 
Synapse 2018 Guarding against failure in a hundred step pipeline
Synapse 2018 Guarding against failure in a hundred step pipelineSynapse 2018 Guarding against failure in a hundred step pipeline
Synapse 2018 Guarding against failure in a hundred step pipelineCalvin French-Owen
 
AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...Amazon Web Services
 
AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...Amazon Web Services
 
Become a Performance Diagnostics Hero
Become a Performance Diagnostics HeroBecome a Performance Diagnostics Hero
Become a Performance Diagnostics HeroTechWell
 
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData
 
SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017Jags Ramnarayan
 
Building Operational Data Lake using Spark and SequoiaDB with Yang Peng
Building Operational Data Lake using Spark and SequoiaDB with Yang PengBuilding Operational Data Lake using Spark and SequoiaDB with Yang Peng
Building Operational Data Lake using Spark and SequoiaDB with Yang PengDatabricks
 
Apache Kafka, and the Rise of Stream Processing
Apache Kafka, and the Rise of Stream ProcessingApache Kafka, and the Rise of Stream Processing
Apache Kafka, and the Rise of Stream ProcessingGuozhang Wang
 
Circonus: Design failures - A Case Study
Circonus: Design failures - A Case StudyCirconus: Design failures - A Case Study
Circonus: Design failures - A Case StudyHeinrich Hartmann
 
London Redshift Meetup - July 2017
London Redshift Meetup - July 2017London Redshift Meetup - July 2017
London Redshift Meetup - July 2017Pratim Das
 
The Future of Real-Time in Spark
The Future of Real-Time in SparkThe Future of Real-Time in Spark
The Future of Real-Time in SparkReynold Xin
 

Similar to hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd (20)

Strata+Hadoop 2015 NYC End User Panel on Real-Time Data Analytics
Strata+Hadoop 2015 NYC End User Panel on Real-Time Data AnalyticsStrata+Hadoop 2015 NYC End User Panel on Real-Time Data Analytics
Strata+Hadoop 2015 NYC End User Panel on Real-Time Data Analytics
 
Flexible and Real-Time Stream Processing with Apache Flink
Flexible and Real-Time Stream Processing with Apache FlinkFlexible and Real-Time Stream Processing with Apache Flink
Flexible and Real-Time Stream Processing with Apache Flink
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
 
Scheduling in Linux and Web Servers
Scheduling in Linux and Web ServersScheduling in Linux and Web Servers
Scheduling in Linux and Web Servers
 
Apache Spark 2.0: Faster, Easier, and Smarter
Apache Spark 2.0: Faster, Easier, and SmarterApache Spark 2.0: Faster, Easier, and Smarter
Apache Spark 2.0: Faster, Easier, and Smarter
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentation
 
Streaming data for real time analysis
Streaming data for real time analysisStreaming data for real time analysis
Streaming data for real time analysis
 
Yogesh kumar kushwah represent’s
Yogesh kumar kushwah represent’sYogesh kumar kushwah represent’s
Yogesh kumar kushwah represent’s
 
Synapse 2018 Guarding against failure in a hundred step pipeline
Synapse 2018 Guarding against failure in a hundred step pipelineSynapse 2018 Guarding against failure in a hundred step pipeline
Synapse 2018 Guarding against failure in a hundred step pipeline
 
AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016: [REPEAT] How EA Leveraged Amazon Redshift and AWS Partner...
 
AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...
AWS re:Invent 2016| GAM301 | How EA Leveraged Amazon Redshift and AWS Partner...
 
Become a Performance Diagnostics Hero
Become a Performance Diagnostics HeroBecome a Performance Diagnostics Hero
Become a Performance Diagnostics Hero
 
CAOS: A CAD Framework for FPGA-Based Systems
CAOS: A CAD Framework for FPGA-Based SystemsCAOS: A CAD Framework for FPGA-Based Systems
CAOS: A CAD Framework for FPGA-Based Systems
 
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
 
SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017
 
Building Operational Data Lake using Spark and SequoiaDB with Yang Peng
Building Operational Data Lake using Spark and SequoiaDB with Yang PengBuilding Operational Data Lake using Spark and SequoiaDB with Yang Peng
Building Operational Data Lake using Spark and SequoiaDB with Yang Peng
 
Apache Kafka, and the Rise of Stream Processing
Apache Kafka, and the Rise of Stream ProcessingApache Kafka, and the Rise of Stream Processing
Apache Kafka, and the Rise of Stream Processing
 
Circonus: Design failures - A Case Study
Circonus: Design failures - A Case StudyCirconus: Design failures - A Case Study
Circonus: Design failures - A Case Study
 
London Redshift Meetup - July 2017
London Redshift Meetup - July 2017London Redshift Meetup - July 2017
London Redshift Meetup - July 2017
 
The Future of Real-Time in Spark
The Future of Real-Time in SparkThe Future of Real-Time in Spark
The Future of Real-Time in Spark
 

More from Michael Stack

hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloudhbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on CloudMichael Stack
 
hbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinteresthbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at PinterestMichael Stack
 
hbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didihbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at DidiMichael Stack
 
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...Michael Stack
 
hbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencenthbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at TencentMichael Stack
 
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...Michael Stack
 
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...Michael Stack
 
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Componenthbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index ComponentMichael Stack
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at AlibabaMichael Stack
 
hbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomihbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at XiaomiMichael Stack
 
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Sparkhbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and SparkMichael Stack
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBaseMichael Stack
 
hbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solutionhbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index SolutionMichael Stack
 
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memoryhbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent MemoryMichael Stack
 
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACLhbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACLMichael Stack
 
hbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBasehbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBaseMichael Stack
 
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...Michael Stack
 
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...Michael Stack
 
HBaseConAsia2019 Keynote
HBaseConAsia2019 KeynoteHBaseConAsia2019 Keynote
HBaseConAsia2019 KeynoteMichael Stack
 
HBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latencies
HBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latenciesHBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latencies
HBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latenciesMichael Stack
 

More from Michael Stack (20)

hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloudhbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
hbaseconasia2019 HBase Table Monitoring and Troubleshooting System on Cloud
 
hbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinteresthbaseconasia2019 Recent work on HBase at Pinterest
hbaseconasia2019 Recent work on HBase at Pinterest
 
hbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didihbaseconasia2019 HBase at Didi
hbaseconasia2019 HBase at Didi
 
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
hbaseconasia2019 The Practice in trillion-level Video Storage and billion-lev...
 
hbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencenthbaseconasia2019 HBase at Tencent
hbaseconasia2019 HBase at Tencent
 
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
hbaseconasia2019 Spatio temporal Data Management based on Ali-HBase Ganos and...
 
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
hbaseconasia2019 Bridging the Gap between Big Data System Software Stack and ...
 
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Componenthbaseconasia2019 Pharos as a Pluggable Secondary Index Component
hbaseconasia2019 Pharos as a Pluggable Secondary Index Component
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
 
hbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomihbaseconasia2019 OpenTSDB at Xiaomi
hbaseconasia2019 OpenTSDB at Xiaomi
 
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Sparkhbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
hbaseconasia2019 BigData NoSQL System: ApsaraDB, HBase and Spark
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
 
hbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solutionhbaseconasia2019 Distributed Bitmap Index Solution
hbaseconasia2019 Distributed Bitmap Index Solution
 
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memoryhbaseconasia2019 HBase Bucket Cache on Persistent Memory
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
 
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACLhbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
hbaseconasia2019 The Procedure v2 Implementation of WAL Splitting and ACL
 
hbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBasehbaseconasia2019 BDS: A data synchronization platform for HBase
hbaseconasia2019 BDS: A data synchronization platform for HBase
 
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
hbaseconasia2019 Further GC optimization for HBase 2.x: Reading HFileBlock in...
 
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
hbaseconasia2019 HBCK2: Concepts, trends, and recipes for fixing issues in HB...
 
HBaseConAsia2019 Keynote
HBaseConAsia2019 KeynoteHBaseConAsia2019 Keynote
HBaseConAsia2019 Keynote
 
HBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latencies
HBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latenciesHBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latencies
HBaseConAsia2018 Track3-1: Serving billions of queries in millisecond latencies
 

Recently uploaded

一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理
一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理
一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理C
 
原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样
原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样
原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样asdafd
 
原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样
原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样
原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样gfhdsfr
 
一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书
一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书
一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书rgdasda
 
一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理A
 
一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理
一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理
一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理Fir
 
一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理
一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理
一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理B
 
一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理
一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理
一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理gfhdsfr
 
🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...
🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...
🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...Mumbai Escorts
 
💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...
💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...
💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...Mumbai Escorts
 
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays SweatshirtsFree on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirtsrahman018755
 
一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理
一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理
一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理Fir
 
Statistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfStatistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfOndejSur
 
Development Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appsDevelopment Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appscristianmanaila2
 
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkkaudience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkklolsDocherty
 
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样Fi
 
Free scottie t shirts Free scottie t shirts
Free scottie t shirts Free scottie t shirtsFree scottie t shirts Free scottie t shirts
Free scottie t shirts Free scottie t shirtsrahman018755
 
一比一定制加州大学欧文分校毕业证学位证书
一比一定制加州大学欧文分校毕业证学位证书一比一定制加州大学欧文分校毕业证学位证书
一比一定制加州大学欧文分校毕业证学位证书A
 
原版定制美国加州大学河滨分校毕业证原件一模一样
原版定制美国加州大学河滨分校毕业证原件一模一样原版定制美国加州大学河滨分校毕业证原件一模一样
原版定制美国加州大学河滨分校毕业证原件一模一样A
 

Recently uploaded (20)

一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理
一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理
一比一原版(Princeton毕业证书)普林斯顿大学毕业证如何办理
 
原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样
原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样
原版定制(Management毕业证书)新加坡管理大学毕业证原件一模一样
 
原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样
原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样
原版定制(爱大毕业证书)英国爱丁堡大学毕业证原件一模一样
 
一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书
一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书
一比一定制(OSU毕业证书)美国俄亥俄州立大学毕业证学位证书
 
一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理
 
一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理
一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理
一比一原版(PSU毕业证书)美国宾州州立大学毕业证如何办理
 
一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理
一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理
一比一原版(Bath毕业证书)英国桑德兰大学毕业证如何办理
 
一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理
一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理
一比一原版(Exon毕业证书)英国埃克塞特大学毕业证如何办理
 
🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...
🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...
🍑👄Dehradun Esℂorts Serviℂe☎️9315791090🍑👄 ℂall Girl serviℂe in ☎️Dehradun ℂall...
 
💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...
💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...
💞 Safe And Seℂure ℂall Girls Dehradun ℂall Girls Serviℂe Just ℂall 🍑👄93157910...
 
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays SweatshirtsFree on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
 
一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理
一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理
一比一原版(NYU毕业证书)美国纽约大学毕业证如何办理
 
Statistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfStatistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdf
 
Development Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appsDevelopment Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of apps
 
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkkaudience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
 
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
 
Free scottie t shirts Free scottie t shirts
Free scottie t shirts Free scottie t shirtsFree scottie t shirts Free scottie t shirts
Free scottie t shirts Free scottie t shirts
 
GOOGLE Io 2024 At takes center stage.pdf
GOOGLE Io 2024 At takes center stage.pdfGOOGLE Io 2024 At takes center stage.pdf
GOOGLE Io 2024 At takes center stage.pdf
 
一比一定制加州大学欧文分校毕业证学位证书
一比一定制加州大学欧文分校毕业证学位证书一比一定制加州大学欧文分校毕业证学位证书
一比一定制加州大学欧文分校毕业证学位证书
 
原版定制美国加州大学河滨分校毕业证原件一模一样
原版定制美国加州大学河滨分校毕业证原件一模一样原版定制美国加州大学河滨分校毕业证原件一模一样
原版定制美国加州大学河滨分校毕业证原件一模一样
 

hbaseconasia2019 Phoenix Practice in China Life Insurance Co., Ltd

  • 1.
  • 2. Phoenix Practice in China Life Insurance Co., Ltd. Leo Yuan yuanliou2015@e-chinalife.com
  • 3. 1 Scenarios 2 Designs 3 Optimizations 4 Problems 5 Future Work Agenda
  • 4. Scenarios - Overview Main Processing Cluster A Application Procession Cluster C Readonly cluster B Sync basic data Sync result Real Time Query Cluster E
  • 5. Scenarios - Overview 4 Cluste200+ Nodes(30+ Phoenix Nodes) Clusters Data 1300TB+ data 30TB+ biggest table Querys Ten million level perday Processing Hundreds of MR/Hive/Spark jobs per day 50TB+ Incremental data for update & insert
  • 6. Scenarios - Overview step 1 get some money from company counter step 2 query operation detail from app This Is What We Do Real Cust Rights View System !
  • 7. Scenarios - Overview 30,000,000 + Incremental data/day 700,000 + users / day 10,000+ records/sec 8,000+ sqls / sec
  • 8. 1 Scenarios 2 Designs 3 Optimizations 4 Problems 5 Future Work Agenda
  • 9. Designs - Data Architecture ü Initialize Data ü Build Phoenix Index ü Sync Real Time Data ü Provide Data Service China Life Insurance APP Business System SharePlex Kafka Spark Streaming Cust View System Phoenix HBase
  • 10. Designs - Development Architecture Data Integration Data View Cross-province Integration Query Distribution Engine V_constract table (4+3) Data Service Ultimate Real Time Query Service Data Exchange Initial Data Sync Program Data Source Business System DataBase JobSchedule Monitor Resourceschedule monitor Schedule Monitor Real Time Data Link Incremental Data Sync Program V_pay table (4+3) V_cust service table (4+3) V_claim table (4+3) … dim Resourceaccess control UserAccess Control Privilege Control
  • 11. Designs - Physical Architecture Phoenix Cluster Configuration DB Weblogic Server Weblogic Server Weblogic Server F5APP Server Phoenix Gateway
  • 12. Kafka Designs - Real Time Data Sync cid c_no type amount branch syssource updtime incr_flag commit_time …… 001 001 M 100 000000 V6 2019-06-11 16:37.322 1 2019-06-11 16:37.322 …… 02 002 R 100 000001 V7 2019-06-11 16:38.689 2 2019-06-11 16:38.689 …… Contract(BeiJing) cid c_no type amount 01 001 M 100 1、Partition By Global Primary Key 2、Shield Upstream System Table Structure Adjustment 3、No Effect on Normal Stream Process when Data Supplement … … …… SparkStreaming Contract(ShangHai) cid c_no type amount 02 002 R 100 SparkStreaming_compt
  • 13. Designs - Ultimate Real Time Query Service label name label type isHolder logic paidBonus sum holder sql1,2,3 paidMoenyList list holder sql4,5 paidExpire sum Insured sql6 … … … … label.properties Analyze Parameters Package Sqls Execue Sqls Collect Results Chinalife Insurance APP Ultimate Real Time Query Service
  • 14. 1 Scenarios 2 Designs 3 Optimizations 4 Problems 5 Future Work Agenda
  • 15. Optimizations - Sql Execution Process DriverManager.getConnection("phoenixUrl") con.prepareStatement(sql) pstat.executeQuery() rs.next() 1、SYSTEM.CATALOG 2、SYSTEM.STATS 3、SYSTEM.LOG 4、 SYSTEM.SEQUENCE 1、Query meta data of table/index from phoenix server(SYSTEM.CATALOG) 2、Determine the table/index sql need to scan 3、Query statistics information of table/index from phoenix server(SYSTEM.STATS) 4、Generate scans based on statistics information、meta data、sql 1、Parallelity decided by phoenix.query.threadPoolSize
  • 16. Optimizations – Phoenix System Table SYSTEM.CATALOG SYSTEM.STATS Describe table/index meta information, such as l TABLE NEME l COLUMN NAME l SALT_BUCKETS l UPDATE_CACHE_FREQUENCY l GUIDE_POST_WIDTH Describe table/index accurate statistics information, such as l GUIDE_POST_KEY l GUIDE_POSTS_WIDTH PS:UPDATE STATISTICS TABLE_NEME
  • 17. Optimizations - RS Group Hmaster SYSTEM:CATALOG SYSTEM:MUTEX SYSTEM:STATS CONTRACT INCOME PERSON RS Group 1 RS Group 2 rs1 SYSTEM:CATALOG SYSTEM:STATS Hmaster rs2 rs3 rs4 … rs1 rs3 rs4 …rs2 …… SYSTEM:STATSCONTRACTINCOME PERSON …… • Metadata Table Isolation to Decrease Impact on Business Table Query
  • 18. Optimizations - UPDATE_CACHE_FREQUENCY • Adjust this parameter to decrease hotspot in SYSTEM.CATALOG p Decide the query frequency of SYSTEM.CATALOG p Default value is “Always” and will Cause read/write pressure in SYSTEM.CATALOG p Can be set per Cluster/Table 1. “phoenix.default.update.cache.frequency”: 86412345 2. create table test.test (a varchar not null primary key,b varchar ) SALT_BUCKETS = 10, UPDATE_CACHE_FREQUENCY=86400000;
  • 19. Optimizations – Salt & Pre-Split • Data table use salt_buckets, Index table use pre-split CREATE TABLE …( )SALT_BUCKETS = 60 CREATE INDEX … ON …( … ) INCLUDE ( … ) ASYNC SALT_BUCKETS = 0 SPLIT ON ( … ) 1、Create index to ensure that all of query process stop at index table 2、use pre split to reduce the chunk number when execute a query 3、use async index to avoid OOM when building the index table 1、Use salted table to distribute data evenly Phoenix Client write Phoenix Client query
  • 20. Optimizations - Open Offheap • Minimal read cost to improve query efficiency BucketCache Configuration Properties l hbase.bucketcache.combinedcache.enabled l hbase.bucketcache.ioengine l hfile.block.cache.size l hbase.bucketcache.size l hbase.bucketcache.bucket.sizes l -XX:MaxDirectMemorySize
  • 21. Optimizations - Other Configurations • Region Balance By Table • G1GC • Manual MajorCompaction
  • 22. 1 Scenarios 2 Designs 3 Optimizations 4 Problems 5 Future Work Agenda
  • 23. Problems - Current time function cause Query performance degradation • This function will lead to client-cluster interaction frequently SELECT … FROM … WHERE date <= CURRENT_TIME() SELECT … FROM … WHERE date <= CURRENT_TIME() SELECT … FROM … WHERE date <= NOW_DATE replace CURRENT_TIME In java code to NOW_DATE ① ② ③
  • 24. Problems - HBase cluster balance abnormal 1.Turn on RSGroup 2.Set two RSGroup default 、my_group 3.Move rs to default and my_group 4.Restart one rs in default process Problems Resolve org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer 2. balance_rsgroup 'default' abnormal 1. balancer abnormal
  • 25. Problems - ACL abnormal when come together with RSGroup 1.Turn on RSGroup 2.Turn on ACL process Problems 1. Non-hbase user can’t creat table Resolve 1, Use hbase user to create table 2, grant this table 'RWX’ to Non-hbase user <property> <name> hbase.coprocessor.master.classes </name> <value> org.apache.hadoop.hbase.security. access.AccessController, org.apache.hadoop. hbase.rsgroup.RSGroupAdminEndpoint </value> </property>
  • 26. 1 Scenarios 2 Designs 3 Optimizations 4 Problems 5 Future Work Agenda
  • 27. Future Work - RPC Read/Write Isolation total queue = hbase.ipc.server.callqueue.handler.factor * handler read queue = total queue * hbase.ipc.server.callqueue.read.ratio write queue = total queue * (1- hbase.ipc.server.callqueue.read.ratio ) scan queue = = total queue * hbase.ipc.server.callqueue.read.ratio * hbase.ipc.server.callqueue.scan.ratio total queue write queue read queue scan queue
  • 28. Future Work - Compaction Contral Set Offpeak Time Set Peak Time Throughput l hbase.offpeak.end.hour l hbase.offpeak.start.hour l hbase.hstore.compaction.throughput.offpeak l key:hbase.regionserver.throughput.controller value:org.apache.hadoop.hbase.regionserver.compactions. PressureAwareCompactionThroughputController Open Compaction Controller l hbase.hstore.compaction.throughput.higher.bound l hbase.hstore.compaction.throughput.lower.bound
  • 29. Future Work - Join Optimization SELECT … FROM ( a1 JOIN (SELECT... FROM b1 WHERE …) a2 ON a1…. = a2…) WHERE … EXPLAIN