Nonrelational Databases
Upcoming SlideShare
Loading in...5
×
 

Nonrelational Databases

on

  • 13,465 views

My improvised/copied preso for some short talk I gave.

My improvised/copied preso for some short talk I gave.

Statistics

Views

Total Views
13,465
Views on SlideShare
13,003
Embed Views
462

Actions

Likes
8
Downloads
221
Comments
0

4 Embeds 462

https://demo.buddycloud.org 439
http://www.slideshare.net 12
http://localhost 9
http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Nonrelational Databases Nonrelational Databases Presentation Transcript

  • Non-relational Databases A new kind of Databases for handling Web Scale
  • Agenda
    • The problem
    • The solution
    • Benefits
    • Cost
    • Example: Cassandra
  • The problem
    • The Web introduces a new scale for applications, in terms of:
      • Concurrent users (millions of reqs/second)
      • Data (peta-bytes generated daily)
      • Processing (all this data needs processing)
      • Exponential growth (surging unpredictable demands)
  • The problem (contd.)
    • Web sites with very large traffic have no way to deal with this using existing RDBMS solutions:
      • Oracle
      • MS SQL
      • Sybase
      • MySQL
      • PostgreSQL
    • Even with their high-end clustering solutions
  • The problem (contd.)
    • Why?
      • Applications using normalized database schema require the use of join's, which doesn't perform well under lots of data and/or nodes
      • Existing RDBMS clustering solutions require scale-up, which is limited & not really scalable when dealing with exponential growth
      • Machines have upper limits on capacity, & sharding the data & processing across machines is very complex & app-specific
  • The problem (contd.)
    • Why not just use sharding?
      • Very problematic when adding/removing nodes
      • Basically, you end up denormalizing everything & loosing all benefits of relational databases
  • Who faced this problem?
    • Web applications dealing with high traffic, massive data, large user-base & user-generated content, such as:
      • Google
      • Yahoo!
      • Amazon
      • Facebook
      • Twitter
      • Linked-In
      • & many more
  • 1 difference though
    • Compared to traditional large applications (telco, financial, &c), these web applications are usually free & therefore:
      • can sacrifice data integrity / consistency
        • No one will sue them if he doesn't receive the most current:
          • status of their friends (Facebook/Twitter)
          • Web search result (Google /Yahoo!)
          • Item added to cart (Amazon)
  • The solution
    • These companies had to come up with a new kind of DBMS, capable of handling web scale
      • Possibly sacrificing some level of consistency or some other feature
  • Must we sacrifice something?
    • In 2000, Eric Brewer (co-founder of Inktomi) formulated the CAP theorem, claiming that you can only optimize 2 out of these 3:
      • C onsistency
      • A vailability
      • P artition-tolerance
    • BTW, the theorem was later proved by MIT scientists in 2002
  • Simple example
    • When you have a lot of data which needs to be highly available, you'll usually need to p artition it across machines & also replicate it to be more fault-tolerant
    • This means, that when writing a record, all replica's must be updated too
    • Now you need to choose between:
      • Lock all relevant replica's during update => be less a vailable
      • Don't lock the replicas => be less c onsistent
  • The consequence
    • You need to either:
      • Drop partition tolerance (CA)
      • Drop availability (CP)
      • Drop consistency (AP)
    • “Drop” here is usually not meant as binary, but rather tunable
  • Non-relational databases
    • The solution these companies came up with are a family of database for handling web scale:
      • BigTable (developed at Google)
      • Hbase (developed at Yahoo!)
      • Dynamo (developed at Amazon)
      • Cassandra (developed at FaceBook)
      • Voldemort (developed at LinkedIn)
      • & a few more:
        • Riak, Redis, CouchDB, MongoDB, Hypertable
  • Benefits
    • Massively scalable
    • Extremely fast
    • Highly available, decentralized & fault tolerant (no single-point-of-failure)
    • Transparent sharding (consistent hashing)
    • Elasticity
    • Parallel processing
    • Dynamic schema
    • Automatic conflict resolution
  • Consistent hashing
  • Replication
  • Replication – node joining
  • Replication – node leaving
  • Scale-out / elasticity?
    • O(1) Distributed Hashtable
    • Runs on a large number of cheap commodity machines
    • Replication
    • Gossip protocol
    • Transparently handles adding/removing nodes
  • Tunable consistency?
    • Levels of consistency:
      • Strict consistency
      • Read your writes consistency
      • Session consistency
      • Monotonic read consistency
      • Eventual consistency
    • Tunable means: how many replica's to lock on write
      • N, R, W parameters
      • Quorum
  • Dealing with inconsistency
    • Read-repair (when encountering inconsistency)
    • Vector clock conflict resolution
  • Dynamic schema
    • Column families (basically a sparse table)
  • Dynamic schema (contd.)
    • “Supercolumn” is a collection of columns
    • Record can have several “supercolumns”
  • Data processing
    • Map/Reduce: an API exposed by non-relational databases to process data
      • A functional programming pattern for parallelizing work
      • Brings the workers to the data – excellent fit for non-relational databases
      • Minimizes the programming to 2 simple functions (map & reduce)
      • Example: count appearances of a word in a giant table of large texts
  • Map/Reduce (contd.)
  • Storage
  • Cost
    • Allows sacrificing consistency (ACID) - at certain circumstances (but can deal with it)
    • Non-standard new API model
    • Non-standard new Schema model
    • New knowledge required to tune/optimize
    • Less mature
  • API model
    • Usually, similar to Key-Value map:
      • Get(key)
      • Put(key, value)
      • Delete(key)
      • Execute(operation, key_list)
    • “value” can be
      • an opaque serialized object
      • a record (list of “columns”: <name, value, timestamp>)
  • Schema model
    • Kind of sparse table
    • No schema
  • Example: Cassandra
    • Features:
      • O(1) DHT
      • Eventual consistency
        • tunable: consistency vs. latency
      • Values are structured, indexed
      • Columns / column families
      • Slicing with predicates (queries)
      • PartitionOrderer
  • Cassandra performance
    • Benchmark against MySQL (50GB)
      • MySQL:
        • 300ms write
        • 350ms read
      • Cassandra:
        • 0.12ms write
        • 15ms read
    • how come writes are so fast?
      • Writes involve no reads/seeks
      • Use any node (closest to you)
  • Cassandra API
  • Cassandra API (contd.)
  • Example: Cassandra (contd.)
    • Java API
      • Simple DAO
      • Simple client
  • Cassandra usage
    • Very high-traffic sites:
      • Facebook
      • Digg
      • Twitter
  • Further information
    • The Dynamo paper:
      • http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
    • Nosql patterns:
      • http://horicky.blogspot.com/2009/11/nosql-patterns.html
    • Nosql conference video's:
      • https://nosqleast.com/2009/
    • Hebrew podcast covering nosql & Cassandra (episodes 56, 57 & more):
      • http://www.reversim.com/
  • Further information (contd.)
    • Ran Tavori's lecture (video + slides):
      • http://prettyprint.me/2010/01/09/introduction-to-nosql-and-cassandra-part-1/
      • http://prettyprint.me/2010/01/20/introduction-to-nosql-and-cassandra-part-2/