HBaseCon2017 Democratizing HBase
Jun. 20, 2017•0 likes•897 views
Download to read offline
Report
Technology
As HBase and Hadoop continue to become routine across enterprises, these enterprises inevitably shift priorities from effective deployments to cost-efficient operations. Consolidation of infrastructure, the sum of hardware, software, and system-administrator effort, is the most common strategy to reduce costs. As a company grows, the number of business organizations, development teams, and individuals accessing HBase grows commensurately, creating a not-so-simple requirement: HBase must effectively service many users, each with a variety of use-cases. This is problem is known as multi-tenancy. While multi-tenancy isn’t a new problem, it also isn’t a solved one, in HBase or otherwise. This talk will present a high-level view of the common issues organizations face when multiple users and teams share a single HBase instance and how certain HBase features were designed specifically to mitigate the issues created by the sharing of finite resources.
HBaseConFollow
Recommended
HBaseCon2017 HBase/Phoenix @ Scale @ SalesforceHBaseCon
introduction-to-apache-kafkaYifeng Jiang
Successes, Challenges, and Pitfalls Migrating a SAAS business to HadoopDataWorks Summit/Hadoop Summit
HBaseCon 2013: Real-Time Model Scoring in Recommender Systems Cloudera, Inc.
Real time fraud detection at 1+M scale on hadoop stackDataWorks Summit/Hadoop Summit
HBaseConAsia2018 Track2-4: HTAP DB-System: AsparaDB HBase, Phoenix, and SparkMichael Stack
More Related Content
What's hot
Spark HBase Connector: Feature Rich and Efficient Access to HBase Through Spa...Databricks
HBaseConAsia2018 Track2-2: Apache Kylin on HBase: Extreme OLAP for big dataMichael Stack
Multitenancy At Bloomberg - HBase and OozieDataWorks Summit
HBaseConAsia2018 Track2-6: Scaling 30TB's of data lake with Apache HBase and ...Michael Stack
How to Use Apache Zeppelin with HWX HDBHortonworks
The Future of Hadoop by Arun Murthy, PMC Apache Hadoop & Cofounder HortonworksData Con LA
Similar to HBaseCon2017 Democratizing HBase
HDF 3.0 IoT Platform for EveryoneYifeng Jiang
Curb Your Insecurity - Tips for a Secure Cluster (with Spark too)!!Pardeep Kumar Mishra (Big Data / Hadoop Consultant)
Meet HBase 2.0 and Phoenix 5.0DataWorks Summit
Hadoop & devOps : better togetherMaxime Lanciaux
Apache Phoenix Query ServerJosh Elser
De-Mystifying the Apache Phoenix QueryServerJosh Elser
More from HBaseCon
hbaseconasia2017: Building online HBase cluster of Zhihu based on KubernetesHBaseCon
hbaseconasia2017: HBase on BeamHBaseCon
hbaseconasia2017: HBase Disaster Recovery Solution at HuaweiHBaseCon
hbaseconasia2017: Removable singularity: a story of HBase upgrade in PinterestHBaseCon
hbaseconasia2017: HareQL：快速HBase查詢工具的發展過程HBaseCon
hbaseconasia2017: Apache HBase at NeteaseHBaseCon
Recently uploaded
Charity Navigator Masterclass - Accountability and Finance BeaconOnBoard
ISO Survey 2022: ISO 27001 certificates (ISMS)Andrey Prozorov, CISM, CIPP/E, CDPSE. LA 27001
class and object in c++.pptxAdarsh College, Hingoli
Datacenter in a Suitcase.pdfMario Fahlandt
AI for Da'wah (Da'wah for AI)Muhammad Angga Muttaqien
Knowledge graph use cases in natural language generationElena Simperl
HBaseCon2017 Democratizing HBase
- 1. Democratizing HBase Josh Elser HBaseCon West June 2017
- 2. 2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Engineer at Hortonworks, Member of the Apache Software Foundation Various roles and involvement in… Top-Level Projects • Accumulo • Calcite • HBase • Phoenix ASF Incubator • Fluo • Gossip • Rya • Slider These Apache project names are trademarks or registered trademarks of the Apache Software Foundation.
- 3. 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Scope Apps Middleware HBase HDFS Hypervisor Linux Hardware
- 4. 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved A Retrospective on Adoption Proof of Concept Single Org Sim-Prod Multi-Org Transition Single Org Transition Multi-Org Sim-Prod Company-Wide Shift 1 User 2+ Users
- 5. 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Optimal Utilization à Tight Budgets à Expensive Hardware Acquisition à Easily Accessible to Clients Bad BadGood
- 6. 6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Multi-tenancy à Out of many workloads, one system. • Low Latency (ms) • Always Up • Guarantees of Multiple 9’s • High Latency (min-hr) • Scheduled/Regular • High Throughput
- 7. 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Our Goals à User isolation à Sharing of resources à Flexibility in resource partitioning à Willful developer ignorance Image: Me, South Philadelphia Graffiti
- 8. 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Security side-step à Strong authentication à No unauthenticated users ÃNo impersonation
- 9. 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Execution Throttles User-Driven Tasks à RPC Quotas à RPC Priorities à Space Quotas HBase Internals à MemStore Flushes à Compactions Image: 3DOM Graffiti
- 10. 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Workload Partitioning à RegionServer Groups – Benefits workload per table à Anti-pattern with present value à Beware external systems à Signal for required improvements Image: Me, South Philadelphia Graffiti
- 11. 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Insight and Metrics à Categorize system metrics à Validate configured limits à Understand HBase/HDFS limitations à Verify application use Image: Me, South Philadelphia Graffiti
- 12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Takeaways à Quantify application needs à Calculate system ability à Meet needs with throttling and partitioning à Understand non-HBase/HDFS processes à Monitor, Interpret, Act
- 13. 13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Thanks! Email: elserj@apache.org Twitter: @joshelser