SlideShare a Scribd company logo
1 of 27
Download to read offline
Intro to HBase
                      Alex Baranau, Sematext International, 2012




Monday, July 9, 12
About Me


                     Software Engineer at Sematext International

                     http://blog.sematext.com/author/abaranau

                     @abaranau

                     http://github.com/sematext (abaranau)




Monday, July 9, 12
Agenda


                     What is HBase?

                     How to use HBase?

                     When to use HBase?




Monday, July 9, 12
What is HBase?




Monday, July 9, 12
What: HBase is...
                     Open-source non-relational distributed
                     column-oriented database modeled after
                     Google’s BigTable.


                       Think of it as a sparse, consistent,
                       distributed, multidimensional, sorted map:

                         labeled tables of rows

                         row consist of key-value cells:

       (row key, column family, column, timestamp) -> value


Monday, July 9, 12
What HBase is NOT
                     Not an SQL database

                     Not relational

                     No joins

                     No fancy query language and no
                     sophisticated query engine

                     No transactions out-of-the box

                     No secondary indices out-of-the box

                     Not a drop-in replacement for your RDBMS


Monday, July 9, 12
What: Features-1

                     Linear scalability, capable of
                     storing hundreds of terabytes of data

                     Automatic and configurable sharding
                     of tables

                     Automatic failover support

                     Strictly consistent reads and writes



Monday, July 9, 12
What: Part of Hadoop
                                 ecosystem

                        Provides realtime random read/write
                        access to data stored in HDFS



                                read          HBase           write

                       Data            read           write             Data
                     Consumer                                         Producer
                                              HDFS            write



Monday, July 9, 12
What: Features-2
                     Integrates nicely with Hadoop MapReduce (both
                     as source and destination)

                     Easy Java API for client access

                     Thrift gateway and REST APIs

                     Bulk import of large amount of data

                     Replication across clusters & backup options

                     Block cache and Bloom filters for real-time
                     queries

                     and many more...



Monday, July 9, 12
How to use HBase?




Monday, July 9, 12
How: the Data
                         Row keys uninterpreted byte arrays

                         Columns grouped in columnfamilies (CFs)

                         CFs defined statically upon table creation

                         Cell is uninterpreted byte array and a timestamp
   Rows are ordered
                                   Different data                    All values stores as
    and accessed by
                                 separated into CFs                      byte arrays
        row key

                       Row Key                                Data
                                                                                            Rows can have
                                         geo:{‘country’:‘Belarus’,‘region’:‘Minsk’}           different
                         Minsk
                                       demography:{‘population’:‘1,937,000’@ts=2011}           columns


                                            geo:{‘country’:‘USA’,‘state’:’NY’}              Cell can have
                     New_York_City     demography:{‘population’:‘8,175,133’@ts=2010,          multiple
                                              ‘population’:‘8,244,910’@ts=2011}               versions

                                                                                             Data can be
                         Suva                         geo:{‘country’:‘Fiji’}
                                                                                            very “sparse”
Monday, July 9, 12
How: Writing the Data
                      Row updates are atomic

                      Updates across multiple rows are NOT
                      atomic, no transaction support out of
                      the box

                      HBase stores N versions of a cell
                      (default 3)

                      Tables are usually “sparse”, not all
                      columns populated in a row


Monday, July 9, 12
How: Reading the Data
                      Reader will always read the last written (and committed)
                      values

                      Reading single row: Get

                      Reading multiple rows: Scan (very fast)

                         Scan usually defines start key and stop key

                         Rows are ordered, easy to do partial key scan

                                   Row Key                  Data
                       ‘login_2012-03-01.00:09:17’    d:{‘user’:‘alex’}
                                     ...                     ...
                       ‘login_2012-03-01.23:59:35’    d:{‘user’:‘otis’}
                       ‘login_2012-03-02.00:00:21’   d:{‘user’:‘david’}


                      Query predicate pushed down via server-side Filters


Monday, July 9, 12
How: MapReduce Integration
                     Out of the box integration with Hadoop
                     MapReduce

                       Data from HBase table can be source
                       for MR job

                       MR job can write data into HBase

                       MR job can write data into HDFS
                       directly and then output files can be
                       very quickly loaded into HBase via
                       “Bulk Loading” functionality


Monday, July 9, 12
How: Sharding the Data
                      Automatic and configurable sharding of
                      tables:

                        Tables partitioned into Regions

                        Region defined by start & end row keys

                        Regions are the “atoms” of
                        distribution

                      Regions are assigned to RegionServers
                      (HBase cluster slaves)



Monday, July 9, 12
How: Setup: Components
                      HBase components


                                              ZooKeeper
                                              ZooKeeper
                                               ZooKeeper


                      client             HMaster
                                          HMaster


                                         RegionServer

                               RegionServer    RegionServer
                                                RegionServer
                                                 RegionServer


Monday, July 9, 12
How: Setup: Hadoop Cluster
                         Typical Hadoop+HBase setup
                                                     Master Node                  HDFS

                                 NameNode      JobTracker                        MapReduce

                                                                                  HBase
                                           HMaster


                         RegionServer         RegionServer                         Slave




                                                                   TaskTracker
           TaskTracker




                                                                                   Nodes

                           DataNode             DataNode



                              Slave Node                     Slave Node
Monday, July 9, 12
How: Setup: Automatic Failover

                     DataNode failures handled by HDFS
                     (replication)

                     RSs failures (incl. caused by whole
                     server failure) handled automatically

                       Master re-assignes Regions to
                       available RSs

                     HMaster failover: automatic with
                     multiple HMasters


Monday, July 9, 12
When to Use HBase?




Monday, July 9, 12
When: What HBase is good at

                     Serving large amount of data: built
                     to scale from the get-go

                     fast random access to the data

                     Write-heavy applications*

                     Append-style writing (inserting/
                     overwriting new data) rather than
                     heavy read-modify-write operations**

      * clients should handle the loss of HTable client-side buffer
      ** see https://github.com/sematext/HBaseHUT


Monday, July 9, 12
When: HBase vs ...


                     Favors consistency over availability

                     Part of a Hadoop ecosystem

                     Great community; adopted by tech
                     giants like Facebook, Twitter,
                     Yahoo!, Adobe, etc.




Monday, July 9, 12
When: Use-cases
                     Audit logging systems

                       track user actions

                       answer questions/queries like:

                         what are the last 10 actions made by
                         user?
                         row key: userId_timestamp

                         which users logged into system
                         yesterday?
                         row key: action_timestamp_userId


Monday, July 9, 12
When: Use-cases

                     Real-time analytics, OLAP

                       real-time counters

                       interactive reports showing
                       trends, breakdowns, etc

                       time-series databases




Monday, July 9, 12
When: Use-cases
                     Monitoring system example




Monday, July 9, 12
When: Use-cases
                     Messages-centered systems

                       twitter-like messages/statuses

                     Content management systems

                       serving content out of HBase

                     Canonical use-case: webtable (pages
                     stored during crawling the web)

                     And others


Monday, July 9, 12
Future


                     Making stable enough to substitute
                     RDBMS in mission critical cases

                     Easier system management

                     Performance improvements




Monday, July 9, 12
Qs?
                     (next: Intro into HBase Internals)




                            Sematext is hiring!
Monday, July 9, 12

More Related Content

What's hot

Etsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureEtsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureDan McKinley
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisArnab Mitra
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 
Hadoop introduction , Why and What is Hadoop ?
Hadoop introduction , Why and What is  Hadoop ?Hadoop introduction , Why and What is  Hadoop ?
Hadoop introduction , Why and What is Hadoop ?sudhakara st
 
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...Simplilearn
 
Seminar Presentation Hadoop
Seminar Presentation HadoopSeminar Presentation Hadoop
Seminar Presentation HadoopVarun Narang
 
Chicago Data Summit: Apache HBase: An Introduction
Chicago Data Summit: Apache HBase: An IntroductionChicago Data Summit: Apache HBase: An Introduction
Chicago Data Summit: Apache HBase: An IntroductionCloudera, Inc.
 
From cache to in-memory data grid. Introduction to Hazelcast.
From cache to in-memory data grid. Introduction to Hazelcast.From cache to in-memory data grid. Introduction to Hazelcast.
From cache to in-memory data grid. Introduction to Hazelcast.Taras Matyashovsky
 
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...Simplilearn
 
Apache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data ProcessingApache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data ProcessingDataWorks Summit
 
Hive + Tez: A Performance Deep Dive
Hive + Tez: A Performance Deep DiveHive + Tez: A Performance Deep Dive
Hive + Tez: A Performance Deep DiveDataWorks Summit
 
Introduction to NoSQL Databases
Introduction to NoSQL DatabasesIntroduction to NoSQL Databases
Introduction to NoSQL DatabasesDerek Stainer
 
Hadoop File system (HDFS)
Hadoop File system (HDFS)Hadoop File system (HDFS)
Hadoop File system (HDFS)Prashant Gupta
 
Introduction to HiveQL
Introduction to HiveQLIntroduction to HiveQL
Introduction to HiveQLkristinferrier
 
Apache Hive Tutorial
Apache Hive TutorialApache Hive Tutorial
Apache Hive TutorialSandeep Patil
 

What's hot (20)

Etsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureEtsy Activity Feeds Architecture
Etsy Activity Feeds Architecture
 
Apache HBase™
Apache HBase™Apache HBase™
Apache HBase™
 
Apache hive introduction
Apache hive introductionApache hive introduction
Apache hive introduction
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
Hadoop introduction , Why and What is Hadoop ?
Hadoop introduction , Why and What is  Hadoop ?Hadoop introduction , Why and What is  Hadoop ?
Hadoop introduction , Why and What is Hadoop ?
 
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...
 
Seminar Presentation Hadoop
Seminar Presentation HadoopSeminar Presentation Hadoop
Seminar Presentation Hadoop
 
Chicago Data Summit: Apache HBase: An Introduction
Chicago Data Summit: Apache HBase: An IntroductionChicago Data Summit: Apache HBase: An Introduction
Chicago Data Summit: Apache HBase: An Introduction
 
Cloudera Hadoop Distribution
Cloudera Hadoop DistributionCloudera Hadoop Distribution
Cloudera Hadoop Distribution
 
From cache to in-memory data grid. Introduction to Hazelcast.
From cache to in-memory data grid. Introduction to Hazelcast.From cache to in-memory data grid. Introduction to Hazelcast.
From cache to in-memory data grid. Introduction to Hazelcast.
 
Big data and Hadoop
Big data and HadoopBig data and Hadoop
Big data and Hadoop
 
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
 
Apache Zookeeper
Apache ZookeeperApache Zookeeper
Apache Zookeeper
 
Apache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data ProcessingApache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data Processing
 
Hive + Tez: A Performance Deep Dive
Hive + Tez: A Performance Deep DiveHive + Tez: A Performance Deep Dive
Hive + Tez: A Performance Deep Dive
 
Introduction to NoSQL Databases
Introduction to NoSQL DatabasesIntroduction to NoSQL Databases
Introduction to NoSQL Databases
 
Hadoop File system (HDFS)
Hadoop File system (HDFS)Hadoop File system (HDFS)
Hadoop File system (HDFS)
 
Introduction to HiveQL
Introduction to HiveQLIntroduction to HiveQL
Introduction to HiveQL
 
Apache Hive Tutorial
Apache Hive TutorialApache Hive Tutorial
Apache Hive Tutorial
 

Similar to Intro to HBase

Intro to HBase Internals & Schema Design (for HBase users)
Intro to HBase Internals & Schema Design (for HBase users)Intro to HBase Internals & Schema Design (for HBase users)
Intro to HBase Internals & Schema Design (for HBase users)alexbaranau
 
Apache HBase - Introduction & Use Cases
Apache HBase - Introduction & Use CasesApache HBase - Introduction & Use Cases
Apache HBase - Introduction & Use CasesData Con LA
 
Hw09 Practical HBase Getting The Most From Your H Base Install
Hw09   Practical HBase  Getting The Most From Your H Base InstallHw09   Practical HBase  Getting The Most From Your H Base Install
Hw09 Practical HBase Getting The Most From Your H Base InstallCloudera, Inc.
 
Introduction To HBase
Introduction To HBaseIntroduction To HBase
Introduction To HBaseAnil Gupta
 
Apache HBase: Introduction to a column-oriented data store
Apache HBase: Introduction to a column-oriented data storeApache HBase: Introduction to a column-oriented data store
Apache HBase: Introduction to a column-oriented data storeChristian Gügi
 
Data Storage and Management project Report
Data Storage and Management project ReportData Storage and Management project Report
Data Storage and Management project ReportTushar Dalvi
 
Facebook keynote-nicolas-qcon
Facebook keynote-nicolas-qconFacebook keynote-nicolas-qcon
Facebook keynote-nicolas-qconYiwei Ma
 
支撑Facebook消息处理的h base存储系统
支撑Facebook消息处理的h base存储系统支撑Facebook消息处理的h base存储系统
支撑Facebook消息处理的h base存储系统yongboy
 
Facebook Messages & HBase
Facebook Messages & HBaseFacebook Messages & HBase
Facebook Messages & HBase强 王
 

Similar to Intro to HBase (20)

Introduction to HBase
Introduction to HBaseIntroduction to HBase
Introduction to HBase
 
Big data and tools
Big data and tools Big data and tools
Big data and tools
 
Intro to HBase Internals & Schema Design (for HBase users)
Intro to HBase Internals & Schema Design (for HBase users)Intro to HBase Internals & Schema Design (for HBase users)
Intro to HBase Internals & Schema Design (for HBase users)
 
H base
H baseH base
H base
 
Hbase
HbaseHbase
Hbase
 
Apache HBase - Introduction & Use Cases
Apache HBase - Introduction & Use CasesApache HBase - Introduction & Use Cases
Apache HBase - Introduction & Use Cases
 
Hw09 Practical HBase Getting The Most From Your H Base Install
Hw09   Practical HBase  Getting The Most From Your H Base InstallHw09   Practical HBase  Getting The Most From Your H Base Install
Hw09 Practical HBase Getting The Most From Your H Base Install
 
Data Storage Management
Data Storage ManagementData Storage Management
Data Storage Management
 
Introduction To HBase
Introduction To HBaseIntroduction To HBase
Introduction To HBase
 
Apache HBase: Introduction to a column-oriented data store
Apache HBase: Introduction to a column-oriented data storeApache HBase: Introduction to a column-oriented data store
Apache HBase: Introduction to a column-oriented data store
 
Data Storage and Management project Report
Data Storage and Management project ReportData Storage and Management project Report
Data Storage and Management project Report
 
Big data hbase
Big data hbase Big data hbase
Big data hbase
 
Facebook keynote-nicolas-qcon
Facebook keynote-nicolas-qconFacebook keynote-nicolas-qcon
Facebook keynote-nicolas-qcon
 
支撑Facebook消息处理的h base存储系统
支撑Facebook消息处理的h base存储系统支撑Facebook消息处理的h base存储系统
支撑Facebook消息处理的h base存储系统
 
Facebook Messages & HBase
Facebook Messages & HBaseFacebook Messages & HBase
Facebook Messages & HBase
 
Hbase
HbaseHbase
Hbase
 
Dsm project-h base-cassandra
Dsm project-h base-cassandraDsm project-h base-cassandra
Dsm project-h base-cassandra
 
Hbase
HbaseHbase
Hbase
 
HBASE Overview
HBASE OverviewHBASE Overview
HBASE Overview
 
Nosql seminar
Nosql seminarNosql seminar
Nosql seminar
 

Recently uploaded

Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 

Recently uploaded (20)

Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 

Intro to HBase

  • 1. Intro to HBase Alex Baranau, Sematext International, 2012 Monday, July 9, 12
  • 2. About Me Software Engineer at Sematext International http://blog.sematext.com/author/abaranau @abaranau http://github.com/sematext (abaranau) Monday, July 9, 12
  • 3. Agenda What is HBase? How to use HBase? When to use HBase? Monday, July 9, 12
  • 5. What: HBase is... Open-source non-relational distributed column-oriented database modeled after Google’s BigTable. Think of it as a sparse, consistent, distributed, multidimensional, sorted map: labeled tables of rows row consist of key-value cells: (row key, column family, column, timestamp) -> value Monday, July 9, 12
  • 6. What HBase is NOT Not an SQL database Not relational No joins No fancy query language and no sophisticated query engine No transactions out-of-the box No secondary indices out-of-the box Not a drop-in replacement for your RDBMS Monday, July 9, 12
  • 7. What: Features-1 Linear scalability, capable of storing hundreds of terabytes of data Automatic and configurable sharding of tables Automatic failover support Strictly consistent reads and writes Monday, July 9, 12
  • 8. What: Part of Hadoop ecosystem Provides realtime random read/write access to data stored in HDFS read HBase write Data read write Data Consumer Producer HDFS write Monday, July 9, 12
  • 9. What: Features-2 Integrates nicely with Hadoop MapReduce (both as source and destination) Easy Java API for client access Thrift gateway and REST APIs Bulk import of large amount of data Replication across clusters & backup options Block cache and Bloom filters for real-time queries and many more... Monday, July 9, 12
  • 10. How to use HBase? Monday, July 9, 12
  • 11. How: the Data Row keys uninterpreted byte arrays Columns grouped in columnfamilies (CFs) CFs defined statically upon table creation Cell is uninterpreted byte array and a timestamp Rows are ordered Different data All values stores as and accessed by separated into CFs byte arrays row key Row Key Data Rows can have geo:{‘country’:‘Belarus’,‘region’:‘Minsk’} different Minsk demography:{‘population’:‘1,937,000’@ts=2011} columns geo:{‘country’:‘USA’,‘state’:’NY’} Cell can have New_York_City demography:{‘population’:‘8,175,133’@ts=2010, multiple ‘population’:‘8,244,910’@ts=2011} versions Data can be Suva geo:{‘country’:‘Fiji’} very “sparse” Monday, July 9, 12
  • 12. How: Writing the Data Row updates are atomic Updates across multiple rows are NOT atomic, no transaction support out of the box HBase stores N versions of a cell (default 3) Tables are usually “sparse”, not all columns populated in a row Monday, July 9, 12
  • 13. How: Reading the Data Reader will always read the last written (and committed) values Reading single row: Get Reading multiple rows: Scan (very fast) Scan usually defines start key and stop key Rows are ordered, easy to do partial key scan Row Key Data ‘login_2012-03-01.00:09:17’ d:{‘user’:‘alex’} ... ... ‘login_2012-03-01.23:59:35’ d:{‘user’:‘otis’} ‘login_2012-03-02.00:00:21’ d:{‘user’:‘david’} Query predicate pushed down via server-side Filters Monday, July 9, 12
  • 14. How: MapReduce Integration Out of the box integration with Hadoop MapReduce Data from HBase table can be source for MR job MR job can write data into HBase MR job can write data into HDFS directly and then output files can be very quickly loaded into HBase via “Bulk Loading” functionality Monday, July 9, 12
  • 15. How: Sharding the Data Automatic and configurable sharding of tables: Tables partitioned into Regions Region defined by start & end row keys Regions are the “atoms” of distribution Regions are assigned to RegionServers (HBase cluster slaves) Monday, July 9, 12
  • 16. How: Setup: Components HBase components ZooKeeper ZooKeeper ZooKeeper client HMaster HMaster RegionServer RegionServer RegionServer RegionServer RegionServer Monday, July 9, 12
  • 17. How: Setup: Hadoop Cluster Typical Hadoop+HBase setup Master Node HDFS NameNode JobTracker MapReduce HBase HMaster RegionServer RegionServer Slave TaskTracker TaskTracker Nodes DataNode DataNode Slave Node Slave Node Monday, July 9, 12
  • 18. How: Setup: Automatic Failover DataNode failures handled by HDFS (replication) RSs failures (incl. caused by whole server failure) handled automatically Master re-assignes Regions to available RSs HMaster failover: automatic with multiple HMasters Monday, July 9, 12
  • 19. When to Use HBase? Monday, July 9, 12
  • 20. When: What HBase is good at Serving large amount of data: built to scale from the get-go fast random access to the data Write-heavy applications* Append-style writing (inserting/ overwriting new data) rather than heavy read-modify-write operations** * clients should handle the loss of HTable client-side buffer ** see https://github.com/sematext/HBaseHUT Monday, July 9, 12
  • 21. When: HBase vs ... Favors consistency over availability Part of a Hadoop ecosystem Great community; adopted by tech giants like Facebook, Twitter, Yahoo!, Adobe, etc. Monday, July 9, 12
  • 22. When: Use-cases Audit logging systems track user actions answer questions/queries like: what are the last 10 actions made by user? row key: userId_timestamp which users logged into system yesterday? row key: action_timestamp_userId Monday, July 9, 12
  • 23. When: Use-cases Real-time analytics, OLAP real-time counters interactive reports showing trends, breakdowns, etc time-series databases Monday, July 9, 12
  • 24. When: Use-cases Monitoring system example Monday, July 9, 12
  • 25. When: Use-cases Messages-centered systems twitter-like messages/statuses Content management systems serving content out of HBase Canonical use-case: webtable (pages stored during crawling the web) And others Monday, July 9, 12
  • 26. Future Making stable enough to substitute RDBMS in mission critical cases Easier system management Performance improvements Monday, July 9, 12
  • 27. Qs? (next: Intro into HBase Internals) Sematext is hiring! Monday, July 9, 12