Co-existence or competition - RDBMS and Hadoop

RDBMS and Hadoop - Co-existence
or competition
Ram Mohan

Copyright © 2011 Flytxt B.V. All rights reserved 1/16/2012

Session Agenda!
 Introduction to RDBMS
 What is Hadoop and Map-Reduce
 Hadoop and RDBMS – A comparison
 Co-Existence – Practical Example - Master Website
 Q&A

Copyright © 2011 Flytxt B.V. All rights reserved 1/16/2012 2

Relational DBMS
 Based on Relational Mathematics principles
 Data is represented in terms of rows and columns of a table
 Relational Terminology
◦ Tuple (Row)
◦ Attribute (Column)
◦ Relation (Table)
 Integrity Constraints
◦ Primary Key
◦ Foreign Key
◦ Alternate Key
 ACID Test
◦ Atomicity
◦ Consistency
◦ Isolation
◦ Durability


Normalization
 Normalization - process of removing data redundancy by decomposing
relations in a Database.
 De normalization - carefully introduced redundancy to improve query
performance.


Relational DBMS


Example Data
S# SNAME STATUS CITY
S1 Smith 20 London
S2 Jones 10 Paris
S3 Blake 30 Paris

P# PNAME COLOR WEIGHT CITY
P1 Nut Red 12 London
P2 Bolt Green 17 Paris
P3 Screw Blue 17 Rome
P4 Screw Red 14 London

S# P# QTY
S1 P1 300
S1 P2 200
S1 P3 400
S2 P1 300
S2 P2 400
S3 P2 200


Five computers & a 640k ;-)

"I think there is a world
market for about five
Moore’s computers"
Law
Thomas Watson 1943,
Chairman of the board of IBM

"640k ought to be enough
for anybody"

Attributed to
Bill Gates in 1981.


The Big Data Challenges
 Sources of Data and the amount of data to analyze is growing
exponentially
 Stale data exists because DW solutions cannot ingest the vast amounts of
data fast enough
 Lack of performance for advanced analytics and complex queries
 The number of users and the concurrency of users is increasing rapidly


Hadoop Architecture


Hadoop – HDFS(Hadoop Distributed File System)
 Reliably store petabytes of replicated data across thousand of nodes
◦ Data divided in to 64 MB blocks, each block replicated three times
 Master/Slave architecture
◦ Master NameNode contains block locations
◦ Slave Datanode manages blocks on local FS
 Built on local commodity hardware
◦ No RAID required


Map-Reduce Model


Hadoop – Limitations
 Is not intended for realtime querying.
 Does not support random access.
 Significant learning curve
 Provides barebones functionality out of the box but scaling is built-in and
inexpensive


Where SQL Makes life easy
 Joining
◦ In a single query, get all products in an order with their product information
 Secondary Indexing
◦ Get CustomerId by e-mail
 Referential Integrity
 Realtime Analysis.
 Millions are trained in SQL and relational data modelling
 RDBMS provides tremendous functionality, but is extremely difficult and
costly to scale


Master Website – A Practical Example


Master Website – RDBMS Use Cases
 Profile Information – That is provided during sign up
 Intelligence generated ie the output of the analytic jobs.
 Any online purchasing track records and account management
 Reporting tools


Master Website – Hadoop Use Cases
 Generating Intelligence from the continuous stream of data
◦ Wall Posts on Facebook
 New tags to be added based on the old logs available, due to new
requirements


A Practical Example – Facebook Architecture


THANK YOU


Co-existence or competition - RDBMS and Hadoop

More Related Content

Similar to Co-existence or competition - RDBMS and Hadoop

More from Flytxt

Recently uploaded

Co-existence or competition - RDBMS and Hadoop

Editor's Notes