Nonrelational Databases

Non-relational Databases A new kind of Databases for handling Web Scale

Agenda
The problem
The solution
Benefits
Cost
Example: Cassandra

The problem
The Web introduces a new scale for applications, in terms of:
Concurrent users (millions of reqs/second)
Data (peta-bytes generated daily)
Processing (all this data needs processing)
Exponential growth (surging unpredictable demands)

The problem (contd.)
Web sites with very large traffic have no way to deal with this using existing RDBMS solutions:
Oracle
MS SQL
Sybase
MySQL
PostgreSQL
Even with their high-end clustering solutions

Why?
Applications using normalized database schema require the use of join's, which doesn't perform well under lots of data and/or nodes
Existing RDBMS clustering solutions require scale-up, which is limited & not really scalable when dealing with exponential growth
Machines have upper limits on capacity, & sharding the data & processing across machines is very complex & app-specific

Why not just use sharding?
Very problematic when adding/removing nodes
Basically, you end up denormalizing everything & loosing all benefits of relational databases

Who faced this problem?
Web applications dealing with high traffic, massive data, large user-base & user-generated content, such as:
Google
Yahoo!
Amazon
Facebook
Twitter
Linked-In
& many more

1 difference though
Compared to traditional large applications (telco, financial, &c), these web applications are usually free & therefore:
can sacrifice data integrity / consistency
No one will sue them if he doesn't receive the most current:
status of their friends (Facebook/Twitter)
Web search result (Google /Yahoo!)
Item added to cart (Amazon)

The solution
These companies had to come up with a new kind of DBMS, capable of handling web scale
Possibly sacrificing some level of consistency or some other feature

Must we sacrifice something?
In 2000, Eric Brewer (co-founder of Inktomi) formulated the CAP theorem, claiming that you can only optimize 2 out of these 3:
C onsistency
A vailability
P artition-tolerance
BTW, the theorem was later proved by MIT scientists in 2002

Simple example
When you have a lot of data which needs to be highly available, you'll usually need to p artition it across machines & also replicate it to be more fault-tolerant
This means, that when writing a record, all replica's must be updated too
Now you need to choose between:
Lock all relevant replica's during update => be less a vailable
Don't lock the replicas => be less c onsistent

The consequence
You need to either:
Drop partition tolerance (CA)
Drop availability (CP)
Drop consistency (AP)
“Drop” here is usually not meant as binary, but rather tunable

Non-relational databases
The solution these companies came up with are a family of database for handling web scale:
BigTable (developed at Google)
Hbase (developed at Yahoo!)
Dynamo (developed at Amazon)
Cassandra (developed at FaceBook)
Voldemort (developed at LinkedIn)
& a few more:
Riak, Redis, CouchDB, MongoDB, Hypertable

Benefits
Massively scalable
Extremely fast
Highly available, decentralized & fault tolerant (no single-point-of-failure)
Transparent sharding (consistent hashing)
Elasticity
Parallel processing
Dynamic schema
Automatic conflict resolution

Consistent hashing

Replication

Replication – node joining

Replication – node leaving

Scale-out / elasticity?
O(1) Distributed Hashtable
Runs on a large number of cheap commodity machines
Replication
Gossip protocol
Transparently handles adding/removing nodes

Tunable consistency?
Levels of consistency:
Strict consistency
Read your writes consistency
Session consistency
Monotonic read consistency
Eventual consistency
Tunable means: how many replica's to lock on write
N, R, W parameters
Quorum

Dealing with inconsistency
Read-repair (when encountering inconsistency)
Vector clock conflict resolution

Dynamic schema
Column families (basically a sparse table)

Dynamic schema (contd.)
“Supercolumn” is a collection of columns
Record can have several “supercolumns”

Data processing
Map/Reduce: an API exposed by non-relational databases to process data
A functional programming pattern for parallelizing work
Brings the workers to the data – excellent fit for non-relational databases
Minimizes the programming to 2 simple functions (map & reduce)
Example: count appearances of a word in a giant table of large texts

Map/Reduce (contd.)

Storage

Cost
Allows sacrificing consistency (ACID) - at certain circumstances (but can deal with it)
Non-standard new API model
Non-standard new Schema model
New knowledge required to tune/optimize
Less mature

API model
Usually, similar to Key-Value map:
Get(key)
Put(key, value)
Delete(key)
Execute(operation, key_list)
“value” can be
an opaque serialized object
a record (list of “columns”: <name, value, timestamp>)

Schema model
Kind of sparse table
No schema

Example: Cassandra
Features:
O(1) DHT
Eventual consistency
tunable: consistency vs. latency
Values are structured, indexed
Columns / column families
Slicing with predicates (queries)
PartitionOrderer

Cassandra performance
Benchmark against MySQL (50GB)
MySQL:
300ms write
350ms read
Cassandra:
0.12ms write
15ms read
how come writes are so fast?
Writes involve no reads/seeks
Use any node (closest to you)

Cassandra API

Cassandra API (contd.)

Example: Cassandra (contd.)
Java API
Simple DAO
Simple client

Cassandra usage
Very high-traffic sites:
Facebook
Digg
Twitter

Further information
The Dynamo paper:
http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
Nosql patterns:
http://horicky.blogspot.com/2009/11/nosql-patterns.html
Nosql conference video's:
https://nosqleast.com/2009/
Hebrew podcast covering nosql & Cassandra (episodes 56, 57 & more):
http://www.reversim.com/

Further information (contd.)
Ran Tavori's lecture (video + slides):
http://prettyprint.me/2010/01/09/introduction-to-nosql-and-cassandra-part-1/
http://prettyprint.me/2010/01/20/introduction-to-nosql-and-cassandra-part-2/

Nonrelational Databases

by dibau_naum_h on Mar 03, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 12

Statistics

Nonrelational Databases — Presentation Transcript