HBaseCon 2013: Using Metrics to Monitor and Debug Apache HBase
Report
Cloudera, Inc.Follow
Jul. 8, 2013•0 likes•5,453 views
HBaseCon 2013: Using Metrics to Monitor and Debug Apache HBase
Jul. 8, 2013•0 likes•5,453 views
Report
Technology
Sports
Presented by: Elliott Clark, Cloudera
Cloudera, Inc.Follow
Recommended
Discoverer online training 10g r2Sumit Tambe
Chef Configuration Management With RightScale - RightScale Compute 2013RightScale
Mastering Puppet Configuration Management in the Cloud - RightScale Compute 2013RightScale
Topup architectureTruong Cuong
Big Data Platform Deployment ArchitectureDavid Ko
ECE544 yashashree waseYashashree Wase
More Related Content
What's hot
20210506 meetingNickHuang49
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016Shannon Williams
20210506 meetingNickHuang49
Load balancerMalarvizhi Subramaniyan
Weblogic Domain Activitysubash prakash
Orchestrate Event-Driven Infrastructure with SaltStackLove Nyberg
Viewers also liked
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...Cloudera, Inc.
HBase Application Performance ImprovementBiju Nair
How to Avoid Pitfalls in Big Data Analytics WebinarDatameer
Spark Streaming& Kafka-The Future of Stream Processing by Hari Shreedharan of...Data Con LA
Sqoop on Spark for Data Ingestion-(Veena Basavaraj and Vinoth Chandar, Uber)Spark Summit
Hadoop Summit 2012 | HBase Consistency and Performance ImprovementsCloudera, Inc.
More from Cloudera, Inc.
Partner Briefing_January 25 (FINAL).pptxCloudera, Inc.
Cloudera Data Impact Awards 2021 - Finalists Cloudera, Inc.
2020 Cloudera Data Impact Awards FinalistsCloudera, Inc.
Edc event vienna presentation 1 oct 2019Cloudera, Inc.
Machine Learning with Limited Labeled Data 4/3/19Cloudera, Inc.
Data Driven With the Cloudera Modern Data Warehouse 3.19.19Cloudera, Inc.
Recently uploaded
How is AI changing journalism? Strategic considerations for publishers and ne...Damian Radcliffe
Mitigating Third-Party Risks: Best Practices for CISOs in Ensuring Robust Sec...TrustArc
Product Research Presentation-Maidy Veloso.pptxMaidyVeloso
Understanding Wireguard, TLS and Workload IdentityChristian Posta
How resolve Gem dependencies in your code?Hiroshi SHIBATA
Common WordPress APIs - Options APIJonathan Bossenger
HBaseCon 2013: Using Metrics to Monitor and Debug Apache HBase
- 1. 1 Using Metrics How to monitor and debug Apache HBase with metrics
- 2. 2 Hi! I’m Elliott eclark@apache.org @elliottnclark Engineer @ Cloudera
- 3. Rules of HBase Metrics 1. Collect Metrics 2. Collect Metrics 3. Know your steady state 4. Explore 5. Correlate 3
- 4. Examples • User Error • Region Skew • GC Pressure • Suboptimal HBase • Balancer not Balancing 4
- 5. Region Skew 5 Photo From: http://www.flickr.com/photos/64618542@N00/
- 7. Region Skew Symptoms • Long tail latencies • Unbalanced WALs • Crashing region servers 7
- 8. Region Skew Fixes • Hash Row Key • Add random salt to start of Row Key • Re-Arrange The Row Key • Add Caching 8
- 9. GC Pressure 9 Photo by: http://www.flickr.com/photos/wwarby/
- 10. GC Pressure 10
- 11. GC Pressure Symptoms • Crashing region servers • ZK Connection Issues • Spiky latency 11
- 12. GC Pressure Fixes • Switch map reduce to Bulk Load • Pre-Aggregate Puts • Reduce data returned per scan#next 12
- 13. Bad Balancer 13 Photo By: http://www.flickr.com/photos/jimmiehomeschoolmom/
- 14. Bad Balancer 14
- 15. Bad Balancer Symptoms • Bad latency after balancer • Bad cache hit on some servers 15
- 16. Coming Soon • Better balancer • More per region metrics • Renamed Metrics • Metrics2 based metrics 16
- 17. Metrics Options Cloudera Manger OpenTSDB 17 Ganglia Ganglia Image From:http://www.flickr.com/photos/hongiiv/
- 18. 18
Editor's Notes
- Slow DOWN!NO THUMBS!
- Talk about the two different catagories.Suboptimal usage of hbaseThings hbase should handle for you.
- Explain that region skew is an example.
- Don’t talk about the symtoms.
- Lots of hot regions on the same region serverRegions fighting over space in the Block Cache.
- Go forth and collect metrics.