The document discusses the democratization of HBase, focusing on optimal utilization and multi-tenancy for better resource sharing and user isolation. It highlights goals like strong security through authentication, execution throttles for user-driven tasks, and the importance of workload partitioning. Key takeaways include the need to quantify application requirements, assess system capabilities, and monitor metrics for effective resource management.

1. Democratizing HBase
Josh Elser
HBaseCon West
June 2017

2. 2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Engineer at Hortonworks, Member of the Apache Software Foundation
Various roles and involvement in…
Top-Level Projects
• Accumulo
• Calcite
• HBase
• Phoenix
ASF Incubator
• Fluo
• Gossip
• Rya
• Slider
These Apache project names are trademarks or registered
trademarks of the Apache Software Foundation.

3. 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Scope
Apps
Middleware
HBase
HDFS
Hypervisor
Linux
Hardware

4. 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
A Retrospective on Adoption
Proof of Concept
Single Org Sim-Prod
Multi-Org Transition
Single Org Transition
Multi-Org Sim-Prod
Company-Wide Shift
1 User
2+ Users

5. 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Optimal Utilization
à Tight Budgets
à Expensive Hardware Acquisition
à Easily Accessible to Clients
Bad BadGood

6. 6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Multi-tenancy
à Out of many workloads, one system.
• Low Latency (ms)
• Always Up
• Guarantees of Multiple 9’s
• High Latency (min-hr)
• Scheduled/Regular
• High Throughput

7. 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Our Goals
à User isolation
à Sharing of resources
à Flexibility in resource partitioning
à Willful developer ignorance
Image: Me, South Philadelphia Graffiti

8. 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Security side-step
à Strong authentication
à No unauthenticated users
ÃNo impersonation

9. 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Execution Throttles
User-Driven Tasks
à RPC Quotas
à RPC Priorities
à Space Quotas
HBase Internals
à MemStore Flushes
à Compactions
Image: 3DOM Graffiti

10. 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Workload Partitioning
à RegionServer Groups
– Benefits workload per table
à Anti-pattern with present value
à Beware external systems
à Signal for required improvements
Image: Me, South Philadelphia Graffiti

11. 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Insight and Metrics
à Categorize system metrics
à Validate configured limits
à Understand HBase/HDFS limitations
à Verify application use
Image: Me, South Philadelphia Graffiti

12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Takeaways
à Quantify application needs
à Calculate system ability
à Meet needs with throttling and partitioning
à Understand non-HBase/HDFS processes
à Monitor, Interpret, Act

13. 13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Thanks!
Email: elserj@apache.org
Twitter: @joshelser