The document evaluates the performance of HBase version 0.20.0 on a small cluster. It describes the testbed setup including hardware specifications and Hadoop/HBase configuration parameters. A series of experiments are run to test random reads, random writes, sequential reads, sequential writes, and scans. The results show significant performance improvements over previous versions, getting closer to the performance levels of Google BigTable as reported in their paper.
At Salesforce, we have deployed many thousands of HBase/HDFS servers, and learned a lot about tuning during this process. This talk will walk you through the many relevant HBase, HDFS, Apache ZooKeeper, Java/GC, and Operating System configuration options and provides guidelines about which options to use in what situation, and how they relate to each other.
HBase 2.0 is the next stable major release for Apache HBase scheduled for early 2017. It is the biggest and most exciting milestone release from the Apache community after 1.0. HBase-2.0 contains a large number of features that is long time in the development, some of which include rewritten region assignment, perf improvements (RPC, rewritten write pipeline, etc), async clients, C++ client, offheaping memstore and other buffers, Spark integration, shading of dependencies as well as a lot of other fixes and stability improvements. We will go into technical details on some of the most important improvements in the release, as well as what are the implications for the users in terms of API and upgrade paths. Existing users of HBase/Phoenix as well as operators managing HBase clusters will benefit the most where they can learn about the new release and the long list of features. We will also briefly cover earlier 1.x release lines and compatibility and upgrade paths for existing users and conclude by giving an outlook on the next level of initiatives for the project.
HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...Cloudera, Inc.
The newly added feature of Coprocessors within HBase allows the application designer to move functionality closer to where the data resides. While this sounds like Stored Procedures as known in the RDBMS realm, they have a different set of properties. The distributed nature of HBase adds to the complexity of their implementation, but the client side API allows for an easy, transparent access to their functionality across many servers. This session explains the concepts behind coprocessors and uses examples to show how they can be used to implement data side extensions to the application code.
At Salesforce, we have deployed many thousands of HBase/HDFS servers, and learned a lot about tuning during this process. This talk will walk you through the many relevant HBase, HDFS, Apache ZooKeeper, Java/GC, and Operating System configuration options and provides guidelines about which options to use in what situation, and how they relate to each other.
HBase 2.0 is the next stable major release for Apache HBase scheduled for early 2017. It is the biggest and most exciting milestone release from the Apache community after 1.0. HBase-2.0 contains a large number of features that is long time in the development, some of which include rewritten region assignment, perf improvements (RPC, rewritten write pipeline, etc), async clients, C++ client, offheaping memstore and other buffers, Spark integration, shading of dependencies as well as a lot of other fixes and stability improvements. We will go into technical details on some of the most important improvements in the release, as well as what are the implications for the users in terms of API and upgrade paths. Existing users of HBase/Phoenix as well as operators managing HBase clusters will benefit the most where they can learn about the new release and the long list of features. We will also briefly cover earlier 1.x release lines and compatibility and upgrade paths for existing users and conclude by giving an outlook on the next level of initiatives for the project.
HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...Cloudera, Inc.
The newly added feature of Coprocessors within HBase allows the application designer to move functionality closer to where the data resides. While this sounds like Stored Procedures as known in the RDBMS realm, they have a different set of properties. The distributed nature of HBase adds to the complexity of their implementation, but the client side API allows for an easy, transparent access to their functionality across many servers. This session explains the concepts behind coprocessors and uses examples to show how they can be used to implement data side extensions to the application code.
Anoop Sam John and Ramkrishna Vasudevan (Intel)
HBase provides an LRU based on heap cache but its size (and so the total data size that can be cached) is limited by Java’s max heap space. This talk highlights our work under HBASE-11425 to allow the HBase read path to work directly from the off-heap area.
In this session, you will learn the work Xiaomi has done to improve the availability and stability of our HBase clusters, including cross-site data and service backup and a coordinated compaction framework. You'll also learn about the Themis framework, which supports cross-row transactions on HBase based on Google's percolator algorithm, and its usage in Xiaomi's applications.
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, SalesforceCloudera, Inc.
The strength of an open source project resides entirely in its developer community; a strong democratic culture of participation and hacking makes for a better piece of software. The key requirement is having developers who are not only willing to contribute, but also knowledgeable about the project’s internal structure and architecture. This session will introduce developers to the core internal architectural concepts of HBase, not just “what” it does from the outside, but “how” it works internally, and “why” it does things a certain way. We’ll walk through key sections of code and discuss key concepts like the MVCC implementation and memstore organization. The goal is to convert serious “HBase Users” into HBase Developer Users”, and give voice to some of the deep knowledge locked in the committers’ heads.
Apache HBase, Accelerated: In-Memory Flush and Compaction HBaseCon
Eshcar Hillel and Anastasia Braginsky (Yahoo!)
Real-time HBase application performance depends critically on the amount of I/O in the datapath. Here we’ll describe an optimization of HBase for high-churn applications that frequently insert/update/delete the same keys, such as for high-speed queuing and e-commerce.
HBaseCon 2012 | HBase and HDFS: Past, Present, Future - Todd Lipcon, ClouderaCloudera, Inc.
Apache HDFS, the file system on which HBase is most commonly deployed, was originally designed for high-latency high-throughput batch analytic systems like MapReduce. Over the past two to three years, the rising popularity of HBase has driven many enhancements in HDFS to improve its suitability for real-time systems, including durability support for write-ahead logs, high availability, and improved low-latency performance. This talk will give a brief history of some of the enhancements from Hadoop 0.20.2 through 0.23.0, discuss some of the most exciting work currently under way, and explore some of the future enhancements we expect to develop in the coming years. We will include both high-level overviews of the new features as well as practical tips and benchmark results from real deployments.
Apache HBase is the Hadoop opensource, distributed, versioned storage manager well suited for random, realtime read/write access. This talk will give an overview on how HBase achieve random I/O, focusing on the storage layer internals. Starting from how the client interact with Region Servers and Master to go into WAL, MemStore, Compactions and on-disk format details. Looking at how the storage is used by features like snapshots, and how it can be improved to gain flexibility, performance and space efficiency.
HBaseCon 2015: HBase at Scale in an Online and High-Demand EnvironmentHBaseCon
Pinterest runs 38 different HBase clusters in production, doing a lot of different types of work—with some doing up to 5 million operations per second. In this talk, you'll get details about how we do capacity planning, maintenance tasks such as online automated rolling compaction, configuration management, and monitoring.
Adobe has packaged HBase in Docker containers and uses Marathon and Mesos to schedule them—allowing us to decouple the RegionServer from the host, express resource requirements declaratively, and open the door for unassisted real-time deployments, elastic (up and down) real-time scalability, and more. In this talk, you'll hear what we've learned and explain why this approach could fundamentally change HBase operations.
Since 2013, Yahoo! has been successfully running multi-tenant HBase clusters. Our tenants run applications ranging from real-time processing (e.g. content personalization, Ad targeting) to operational warehouses (e.g. advertising, content). Tenants are guaranteed an adequate level of resource isolation and security. This is achieved through the use of open source and in-house developed HBase features such as region server groups, group-based replication, and group-based favored nodes.
Today, with the increase in adoption and new use cases, we are working towards scaling our HBase clusters to support petabytes of data without compromising on performance and operability. A common tradeoff when scaling a cluster to this size is to increase the size of a region, thus avoiding the problem of having too many regions on a cluster. However, large regions negatively affect the performance and operability of a cluster mainly because region size determines the following: 1. granularity for load distribution, and 2. amount of write amplification due to compaction. Thus we are working towards enabling an HBase cluster to host at least a million regions.
In this presentation, we will walk through the key features we have implemented as well as share our experiences working on multi-tenancy and scaling the cluster.
This talk delves into the many ways that a user has to use HBase in a project. Lars will look at many practical examples based on real applications in production, for example, on Facebook and eBay and the right approach for those wanting to find their own implementation. He will also discuss advanced concepts, such as counters, coprocessors and schema design.
Now that you've seen Base 1.0, what's ahead in HBase 2.0, and beyond—and why? Find out from this panel of people who have designed and/or are working on 2.0 features.
Anoop Sam John and Ramkrishna Vasudevan (Intel)
HBase provides an LRU based on heap cache but its size (and so the total data size that can be cached) is limited by Java’s max heap space. This talk highlights our work under HBASE-11425 to allow the HBase read path to work directly from the off-heap area.
In this session, you will learn the work Xiaomi has done to improve the availability and stability of our HBase clusters, including cross-site data and service backup and a coordinated compaction framework. You'll also learn about the Themis framework, which supports cross-row transactions on HBase based on Google's percolator algorithm, and its usage in Xiaomi's applications.
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, SalesforceCloudera, Inc.
The strength of an open source project resides entirely in its developer community; a strong democratic culture of participation and hacking makes for a better piece of software. The key requirement is having developers who are not only willing to contribute, but also knowledgeable about the project’s internal structure and architecture. This session will introduce developers to the core internal architectural concepts of HBase, not just “what” it does from the outside, but “how” it works internally, and “why” it does things a certain way. We’ll walk through key sections of code and discuss key concepts like the MVCC implementation and memstore organization. The goal is to convert serious “HBase Users” into HBase Developer Users”, and give voice to some of the deep knowledge locked in the committers’ heads.
Apache HBase, Accelerated: In-Memory Flush and Compaction HBaseCon
Eshcar Hillel and Anastasia Braginsky (Yahoo!)
Real-time HBase application performance depends critically on the amount of I/O in the datapath. Here we’ll describe an optimization of HBase for high-churn applications that frequently insert/update/delete the same keys, such as for high-speed queuing and e-commerce.
HBaseCon 2012 | HBase and HDFS: Past, Present, Future - Todd Lipcon, ClouderaCloudera, Inc.
Apache HDFS, the file system on which HBase is most commonly deployed, was originally designed for high-latency high-throughput batch analytic systems like MapReduce. Over the past two to three years, the rising popularity of HBase has driven many enhancements in HDFS to improve its suitability for real-time systems, including durability support for write-ahead logs, high availability, and improved low-latency performance. This talk will give a brief history of some of the enhancements from Hadoop 0.20.2 through 0.23.0, discuss some of the most exciting work currently under way, and explore some of the future enhancements we expect to develop in the coming years. We will include both high-level overviews of the new features as well as practical tips and benchmark results from real deployments.
Apache HBase is the Hadoop opensource, distributed, versioned storage manager well suited for random, realtime read/write access. This talk will give an overview on how HBase achieve random I/O, focusing on the storage layer internals. Starting from how the client interact with Region Servers and Master to go into WAL, MemStore, Compactions and on-disk format details. Looking at how the storage is used by features like snapshots, and how it can be improved to gain flexibility, performance and space efficiency.
HBaseCon 2015: HBase at Scale in an Online and High-Demand EnvironmentHBaseCon
Pinterest runs 38 different HBase clusters in production, doing a lot of different types of work—with some doing up to 5 million operations per second. In this talk, you'll get details about how we do capacity planning, maintenance tasks such as online automated rolling compaction, configuration management, and monitoring.
Adobe has packaged HBase in Docker containers and uses Marathon and Mesos to schedule them—allowing us to decouple the RegionServer from the host, express resource requirements declaratively, and open the door for unassisted real-time deployments, elastic (up and down) real-time scalability, and more. In this talk, you'll hear what we've learned and explain why this approach could fundamentally change HBase operations.
Since 2013, Yahoo! has been successfully running multi-tenant HBase clusters. Our tenants run applications ranging from real-time processing (e.g. content personalization, Ad targeting) to operational warehouses (e.g. advertising, content). Tenants are guaranteed an adequate level of resource isolation and security. This is achieved through the use of open source and in-house developed HBase features such as region server groups, group-based replication, and group-based favored nodes.
Today, with the increase in adoption and new use cases, we are working towards scaling our HBase clusters to support petabytes of data without compromising on performance and operability. A common tradeoff when scaling a cluster to this size is to increase the size of a region, thus avoiding the problem of having too many regions on a cluster. However, large regions negatively affect the performance and operability of a cluster mainly because region size determines the following: 1. granularity for load distribution, and 2. amount of write amplification due to compaction. Thus we are working towards enabling an HBase cluster to host at least a million regions.
In this presentation, we will walk through the key features we have implemented as well as share our experiences working on multi-tenancy and scaling the cluster.
This talk delves into the many ways that a user has to use HBase in a project. Lars will look at many practical examples based on real applications in production, for example, on Facebook and eBay and the right approach for those wanting to find their own implementation. He will also discuss advanced concepts, such as counters, coprocessors and schema design.
Now that you've seen Base 1.0, what's ahead in HBase 2.0, and beyond—and why? Find out from this panel of people who have designed and/or are working on 2.0 features.
Hadoop has proven to be an invaluable tool for many companies over the past few years. Yet it has it's ways and knowing them up front can safe valuable time. This session is a run down of the ever recurring lessons learned from running various Hadoop clusters in production since version 0.15.
What to expect from Hadoop - and what not? How to integrate Hadoop into existing infrastructure? Which data formats to use? What compression? Small files vs big files? Append or not? Essential configuration and operations tips. What about querying all the data? The project, the community and pointers to interesting projects that complement the Hadoop experience.
(SDD401) Amazon Elastic MapReduce Deep Dive and Best Practices | AWS re:Inven...Amazon Web Services
Amazon Elastic MapReduce is one of the largest Hadoop operators in the world. Since its launch five years ago, AWS customers have launched more than 5.5 million Hadoop clusters. In this talk, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters and other Amazon EMR architectural patterns. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We also share best practices to keep your Amazon EMR cluster cost efficient.
This presentation was given to the system adminstration team to give them an idea of how GC works and what to look for when there is abottleneck and troubles.
Revisiting CephFS MDS and mClock QoS SchedulerYongseok Oh
This presents the CephFS performance scalability and evaluation results. Specifically, it addresses some technical issues such as multi core scalability, cache size, static pinning, recovery, and QoS.
Introduction to Sqoop Aaron Kimball Cloudera Hadoop User Group UKSkills Matter
In this talk of Hadoop User Group UK meeting, Aaron Kimball from Cloudera introduces Sqoop, the open source SQL-to-Hadoop tool. Sqoop helps users perform efficient imports of data from RDBMS sources to Hadoop's distributed file system, where it can be processed in concert with other data sources. Sqoop also allows users to export Hadoop-generated results back to an RDBMS for use with other data pipelines.
After this session, users will understand how databases and Hadoop fit together, and how to use Sqoop to move data between these systems. The talk will provide suggestions for best practices when integrating Sqoop and Hadoop in your data processing pipelines. We'll also cover some deeper technical details of Sqoop's architecture, and take a look at some upcoming aspects of Sqoop's development roadmap.
This is a copy of the NoSQL Day 2019 session presented in Washington D.C on May 2019. It covers a series of the most common HBase issues observed among Cloudera customer base, together with RCA and recipes for recovery.
With the advent of Hadoop, there comes the need for professionals skilled in Hadoop Administration making it imperative to be skilled as a Hadoop Admin for better career, salary and job opportunities.
Processing massive amount of data with Map Reduce using Apache Hadoop - Indi...IndicThreads
Session presented at the 2nd IndicThreads.com Conference on Cloud Computing held in Pune, India on 3-4 June 2011.
http://CloudComputing.IndicThreads.com
Abstract: The processing of massive amount of data gives great insights into analysis for business. Many primary algorithms run over the data and gives information which can be used for business benefits and scientific research. Extraction and processing of large amount of data has become a primary concern in terms of time, processing power and cost. Map Reduce algorithm promises to address the above mentioned concerns. It makes computing of large sets of data considerably easy and flexible. The algorithm offers high scalability across many computing nodes. This session will introduce Map Reduce algorithm, followed by few variations of the same and also hands on example in Map Reduce using Apache Hadoop.
Speaker: Allahbaksh Asadullah is a Product Technology Lead from Infosys Labs, Bangalore. He has over 5 years of experience in software industry in various technologies. He has extensively worked on GWT, Eclipse Plugin development, Lucene, Solr, No SQL databases etc. He speaks at the developer events like ACM Compute, Indic Threads and Dev Camps.
Similar to HBase 0.20.0 Performance Evaluation (20)
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
HBase 0.20.0 Performance Evaluation
1. HBase‐0.20.0 Performance Evaluation
Anty Rao and Schubert Zhang, August 21, 2009
{ant.rao, schubert.zhang}@gmail.com,http://cloudepr.blogspot.com/
Background
We have been using HBase for around a year in our development and projects, from 0.17.x to
0.19.x. We and all in the community know the critical performance and reliability issues of these
releases.
Now, the great news is that HBase‐0.20.0 will be released soon. Jonathan Gray from Streamy,
Ryan Rawson from StumbleUpon, Michael Stack from Powerset/Microsoft, Jean‐Daniel Cryans
from OpenPlaces, and other contributors had done a great job to redesign and rewrite many
codes to promote HBase. The two presentations [1] [2] provide more details of this release.
The primary themes of HBase‐0.20.0:
− Performance
Real‐time and Unjavafy software implementations.
HFile, based on BigTable’s SSTable. New file format limits index size.
New API
New Scanners
New Block Cache
Compression (LZO, GZ)
Almost a RegionServer rewrite
− ZooKeeper integration, multiple masters (partly, 0.21 will rewrite Master with better ZK
integration)
Then, we will get a bran‐new, high performance (Random Access, Scan, Insert, …), and stronger
HBase. HBase‐0.20.0 shall be a great milestone, and we should say thanks to all developers.
Following items are very important for us:
− Insert performance: We have very big datasets, which are generated fast.
− Scan performance: For data analysis in MapReduce framework.
− Random Access performance: Provide real‐time services.
− Less memory and I/O overheads: We are neither Google nor Yahoo!, we cannot operate
big cluster with hundreds or thousands of machines, but we really have big data.
− The HFile: Same as SSTable. It should be a common and effective storage element.
Testbed Setup
Cluster:
− 4 slaves + 1 master
− Machine: 4 CPU cores (2.0G), 2x500GB 7200RPM SATA disks, 8GB RAM.
− Linux: RedHat 5.1 (2.6.18‐53.el5), ext3, no RAID
1
2. − 1Gbps network, all nodes under the same switch.
− Hadoop‐0.20.0 (1GB heap), HBase‐0.20.0 RC2 (4GB heap), Zookeeper‐3.2.0
By now, HBase‐0.20.0 RC2 is available for evaluation:
http://people.apache.org/~stack/hbase‐0.20.0‐candidate‐2/. Refer to [2] and [4] for installation.
Hadoop‐0.20.0 configuration important parameters:
core‐site.xml:
parameter value notes
io.file.buffer.size 65536 Irrelevant to this evaluation. We like use this size to improve
file I/O. [5]
io.seqfile.compress.blocksize 327680 Irrelevant to this evaluation. We like use this size to compress
data block.
hdfs‐site.xml
parameter value notes
dfs.namenode.handler.count 20 [5]
dfs.datanode.handler.count 20 [5] [6]
dfs.datanode.max.xcievers 3072 Under heavy read load, you may see lots of DFSClient
complains about no live nodes hold a particular block. [6]
dfs.replication 2 Our cluster is very small. 2 replicas are enough.
mapred‐site.xml
parameter value notes
io.sort.factor 25 [5]
io.sort.mb 250 10 * io.sort.factor [5]
mapred.tasktracker.map.tasks.maximum 3 Since our cluster is very small, we
set 3 to avoid client side bottleneck.
mapred.child.java.opts ‐Xmx512m JVM GC option
‐XX:+UseConcMarkSweepGC
mapred.reduce.parallel.copies 20
mapred.job.tracker.handler.count 10 [5]
hadoop‐env.sh
parameter value notes
HADOOP_CLASSPATH ${HADOOP_HOME}/../hbase‐0.20.0/hbase‐0.20.0.jar: Use HBase jar and
${HADOOP_HOME}/../hbase‐0.20.0/conf:${HADOOP_ configurations. Use
HOME}/../hbase‐0.20.0/lib/zookeeper‐r785019‐hbas Zookeeper jar.
e‐1329.jar
HADOOP_OPTS "‐server ‐XX:+UseConcMarkSweepGC" JVM GC option
HBase‐0.20.0 configuration important parameters:
hbase‐site.xml
2
3. parameter value notes
hbase.cluster.distributed true Fully‐distributed with unmanaged ZooKeeper Quorum
hbase.regionserver.handler.count 20
hbase‐env.sh
parameter value notes
HBASE_CLASSPATH ${HBASE_HOME}/../hadoop‐0.20.0/conf Use hadoop configurations.
HBASE_HEAPSIZE 4000 Give HBase enough heap size
HBASE_OPTS "‐server ‐XX:+UseConcMarkSweepGC" JVM GC option
HBASE_MANAGES_ZK false Refers to hbase.cluster.distributed
Performance Evaluation Programs
We modified the class org.apache.hadoop.hbase.PerformanceEvaluation, since the code has
following problems: The patch is available at http://issues.apache.org/jira/browse/HBASE‐1778.
− It is not updated according to hadoop‐0.20.0.
− The approach to split maps is not strict. Need to provide correct InputSplit and
InputFormat classes. Current code uses TextInputFormat and FileSplit, it is not
reasonable.
The evaluation programs use MapReduce to do parallel operations against an HBase table.
− Total rows: 4,194,280.
− Row size: 1000 bytes for value, and 10 bytes for rowkey. Then we have ~4GB of data.
− Sequential row ranges: 40. (It’s also used to define the total number of MapTasks in
each evaluation.)
− Rows of each sequential range: 104,857
The principle is same as the evaluation programs described in Section 7, Performance Evaluation,
of the Google BigTable paper [3], pages 8‐10. Since we have only 4 nodes to work as clients, we
set mapred.tasktracker.map.tasks.maximum=3 to avoid client side bottleneck.
Performance Evaluation
Report‐1: Normal, autoFlush=false, writeToWAL=true
Google paper
Eventual Total
rows/s ms/row (rows/s in
Experiment Elapsed row/s throughput
per node per node single node
Time(s) (MB/s)
cluster)
random reads 948 4424 1106 0.90 4.47 1212
random writes(i) 390 10755 2689 0.37 10.86 8850
random writes 370 11366 2834 0.35 11.45 8850
sequential reads 193 21732 5433 0.18 21.95 4425
sequential writes(i) 359 11683 2921 0.34 11.80 8547
sequential writes 204 20560 5140 0.19 20.77 8547
scans 68 61681 15420 0.06 62.30 15385
3
4. Report‐2: Normal, autoFlush=true, writeToWAL=true
Google paper
Eventual Total
rows/s ms/row (rows/s in
Experiment Elapsed row/s throughput
per node per node single node
Time(s) (MB/s)
cluster)
sequential writes(i) 461 9098 2275 0.44 9.19 8547
sequential writes 376 11155 2789 0.36 11.27 8547
random writes 600 6990 1748 0.57 7.06 8850
Random writes (i) and sequential write (i) are operations against a new table, i.e. initial writes.
Random writes and sequential writes are operations against an existing table that is already
distributed on all region servers, i.e. distributed writes. Since there is only one region server is
accessed at the beginning of inserting, the performance is not so good until the regions are split
across all region servers. The difference between initial writes and distributed writes for random
writes is not so obvious, but that is distinct for sequential writes.
In Google’s BigTable paper [3], the performance of random writes and sequential writes is very
close. But in our result, sequential writes are faster than random writes, it seems the client side
write‐buffer (12MB, autoFlush=false) taking effect, since it can reduce the number of RPC
packages. And it may imply that further RPC optimizations can gain better performance. If we set
autoFlush true, they will be close too, and less than the number of random writes in report‐1. The
report‐2 shows our inference is correct.
Random reads are far slower than all other operations. Each random read involves the transfer of
a 64KB HFlie block from HDFS to a region server, out of which only a single 1000‐byte value is
used. So the real throughput is approximately 1106*64KB=70MB/s of data read from HDFS, it is
not low. [3] [8]
The average time to random read a row is sub‐ms here (0.90ms) on average per node, that seems
about as good as we can get on our hardware. We're already showing 10X better performance
than a disk seeking (10ms). It should be major contributed by the new BlockCache and new HFile
implementations. Any other improvements will have to come from HDFS optimizations, RPC
optimizations, and of course we can always get better performance by loading up with more RAM
for the file‐system cache. Try 16GB or more RAM, we might get greater performance. But
remember, we're serving out of memory and not disk seeking. Adding more memory (and region
server heap) should help the numbers across the board. The BigTable paper [3] shows 1212
random reads per second on a single node. That's sub‐ms for random access, it’s clearly not
actually doing disk seeks for most gets. (Thanks for good comments from Jonathan Gray and
Michael Stack.) [9]
The performance of sequential reads and scans is very good here, even better than the Google
paper’s [3]. We believe this performance will greatly support HBase for data analysis
(MapReduce). In Google’s BigTable paper [3], the performance of writes will be better than reads,
includes sequential reads. But in our test result, the sequential reads are better than writes.
Maybe there are rooms to improve the performance of writes in the future.
4
5.
And remember, the dataset size in our tests is not big enough (only average 1GB per node), so the
hit ratio of BlockCache is very high (>40%). If a region server serves large dataset (e.g. 1TB), the
power of BlockCache would be downgraded.
Bloom filters can reduce the unnecessary search in HFiles, which can speed up reads, especially
for large datasets when there are multiple files in an HBase region.
We can also consider RAID0 on multiple disks, and mount local file system with noatime and
nodiratime options, for performance improvements.
Performance Evaluation (none WAL)
In some use cases, such as bulk loading a large dataset into an HBase table, the overhead of the
Write‐Ahead‐Logs (commit‐logs) are considerable, since the bulk inserting causes the logs get
rotated often and produce many disk I/O. Here we consider to disable WAL in such use cases, but
the risk is data loss when region server crash. Here I cite a post of Jean‐Daniel Cryans on HBase
mailing list [7].
“As you may know, HDFS still does not support appends. That means that the write ahead logs or WAL
that HBase uses are only helpful if synced on disk. That means that you lose some data during a region
server crash or a kill ‐9. In 0.19 the logs could be opened forever if they had under 100000 edits. Now in 0.20
we fixed that by capping the WAL to ~62MB and we also rotate the logs after 1 hour. This is all good because
it means far less data loss until we are able to append to files in HDFS.
Now to why this may slow down your import, the job I was talking about had huge rows so the logs
got rotated much more often whereas in 0.19 only the number of rows triggered a log rotation. Not writing
to the WAL has the advantage of using far less disk IO but, as you can guess, it means huge data loss in the
case of a region server crash. But, in many cases, a RS crash still means that you must restart your job
because log splitting can take more than 10 minutes so many tasks times out (I am currently working on that
for 0.21 to make it really faster btw).”
So, here we call put.setWriteToWAL(false) to disable WAL, and expect get better writing
performance. This table is the evaluation result.
Report‐3: NonWAL (autoFlush=false, writeToWAL=false)
Google paper
Eventual Total
rows/s ms/row (rows/s in
Experiment Elapsed row/s throughput
per node per node single node
Time(s) (MB/s)
cluster)
random reads 1001 4190 1048 0.95 4.23 1212
random writes(i) 260 16132 4033 0.25 16.29 8850
random writes 194 21620 5405 0.19 21.84 8850
sequential reads 187 22429 5607 0.18 22.65 4425
sequential writes(i) 241 17404 4351 0.23 17.58 8547
sequential writes 122 34379 8595 0.12 34.72 8547
scans 62 67650 16912 0.06 68.33 15385
5
6.
We can see the performance of sequential writes and random writes are far better (~double)
than the normal case (with WAL). So, we can consider using this method to solve some bulk
loading problems which need high performance for inserting. But we suggest calling admin.flush()
to flush the data in memstores to HDFS, immediately after each bulk loading job, to persist data
and avoid loss as much as possible. The above evaluation does not include the time of flush.
Conclusions
Compares to the metrics in Google’s BigTable paper [3], the write performance is still not so good,
but this result is much better than any previous HBase release, especially for the random reads.
We even got better result than the paper [3] on sequential reads and scans. This result gives us
more confidence.
HBase‐0.20.0 should be good to be used:
− as a data store to be inserted/loaded large datasets fast
− to store large datasets to be analyzed by MapReduce jobs
− to provide real‐time query services
We have made a comparison of the performance to run MapReduce jobs which sequentially
retrieve or scan data from HBase tables, and from HDFS files. The latter is trebly faster. And the
gap is even bigger for sequential writes.
HBase‐0.20.0 is not the final word on performance and features. We are looking forward to and
researching following features, and need to read code detail.
− Other RPC‐related improvements
− Other Java‐related improvements
− New master implementation
− Bloom Filter
− Bulk‐load [10]
In the world of big data, best performance usually comes from tow primary schemes:
(1) Reducing disk I/O and disk seek.
If we come back to review the BigTable paper [3] Section 6, Refinements, we can find the
goals of almost all those refinements are related to reducing disk I/O and disk seek, such as
Locality Group, Compression, Caching (Scan Cache and Block Cache), Bloom filters, Group
commit, etc.
(2) Sequential data access.
When implementing our applications and organize/store our big data, we should always try
our best to write and read data in sequential mode. Maybe sometimes we cannot find a
generalized access scheme like that in the traditional database world, to access data in
various views, but this may be the trait of big data world.
6
7. References:
[1] Ryan Rawson’s Presentation on NOSQL.
http://blog.oskarsson.nu/2009/06/nosql‐debrief.html
[2] HBase goes Realtime, The HBase presentation at HadoopSummit2009 by Jonathan Gray
and Jean‐Daniel Cryans
http://wiki.apache.org/hadoop‐data/attachments/HBase(2f)HBasePresentations/attach
ments/HBase_Goes_Realtime.pdf
[3] Google paper, Bigtable: A Distributed Storage System for Structured Data
http://labs.google.com/papers/bigtable.html
[4] HBase‐0.20.0‐RC2 Documentation,
http://people.apache.org/~stack/hbase‐0.20.0‐candidate‐2/docs/
[5] Cloudera, Hadoop Configuration Parameters.
http://www.cloudera.com/blog/category/mapreduce/page/2/
[6] HBase Troubleshooting, http://wiki.apache.org/hadoop/Hbase/Troubleshooting
[7] Jean‐Daniel Cryans’s post on HBase mailing list, on Aug 12, 2009: Tip when migrating
your data loading MR jobs from 0.19 to 0.20.
[8] ACM Queue, Adam Jacobs, 1010data Inc., The Pathologies of Big Data,
http://queue.acm.org/detail.cfm?id=1563874
[9] Our another evaluation report:
http://docloud.blogspot.com/2009/08/hbase‐0200‐performance‐evaluation.html
[10] Yahoo! Research, Efficient Bulk Insertion into a Distributed Ordered Table,
http://research.yahoo.com/files/bulkload.pdf
7