• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Nonrelational Databases
 

Nonrelational Databases

on

  • 13,275 views

My improvised/copied preso for some short talk I gave.

My improvised/copied preso for some short talk I gave.

Statistics

Views

Total Views
13,275
Views on SlideShare
12,813
Embed Views
462

Actions

Likes
8
Downloads
219
Comments
0

4 Embeds 462

https://demo.buddycloud.org 439
http://www.slideshare.net 12
http://localhost 9
http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Nonrelational Databases Nonrelational Databases Presentation Transcript

    • Non-relational Databases A new kind of Databases for handling Web Scale
    • Agenda
      • The problem
      • The solution
      • Benefits
      • Cost
      • Example: Cassandra
    • The problem
      • The Web introduces a new scale for applications, in terms of:
        • Concurrent users (millions of reqs/second)
        • Data (peta-bytes generated daily)
        • Processing (all this data needs processing)
        • Exponential growth (surging unpredictable demands)
    • The problem (contd.)
      • Web sites with very large traffic have no way to deal with this using existing RDBMS solutions:
        • Oracle
        • MS SQL
        • Sybase
        • MySQL
        • PostgreSQL
      • Even with their high-end clustering solutions
    • The problem (contd.)
      • Why?
        • Applications using normalized database schema require the use of join's, which doesn't perform well under lots of data and/or nodes
        • Existing RDBMS clustering solutions require scale-up, which is limited & not really scalable when dealing with exponential growth
        • Machines have upper limits on capacity, & sharding the data & processing across machines is very complex & app-specific
    • The problem (contd.)
      • Why not just use sharding?
        • Very problematic when adding/removing nodes
        • Basically, you end up denormalizing everything & loosing all benefits of relational databases
    • Who faced this problem?
      • Web applications dealing with high traffic, massive data, large user-base & user-generated content, such as:
        • Google
        • Yahoo!
        • Amazon
        • Facebook
        • Twitter
        • Linked-In
        • & many more
    • 1 difference though
      • Compared to traditional large applications (telco, financial, &c), these web applications are usually free & therefore:
        • can sacrifice data integrity / consistency
          • No one will sue them if he doesn't receive the most current:
            • status of their friends (Facebook/Twitter)
            • Web search result (Google /Yahoo!)
            • Item added to cart (Amazon)
    • The solution
      • These companies had to come up with a new kind of DBMS, capable of handling web scale
        • Possibly sacrificing some level of consistency or some other feature
    • Must we sacrifice something?
      • In 2000, Eric Brewer (co-founder of Inktomi) formulated the CAP theorem, claiming that you can only optimize 2 out of these 3:
        • C onsistency
        • A vailability
        • P artition-tolerance
      • BTW, the theorem was later proved by MIT scientists in 2002
    • Simple example
      • When you have a lot of data which needs to be highly available, you'll usually need to p artition it across machines & also replicate it to be more fault-tolerant
      • This means, that when writing a record, all replica's must be updated too
      • Now you need to choose between:
        • Lock all relevant replica's during update => be less a vailable
        • Don't lock the replicas => be less c onsistent
    • The consequence
      • You need to either:
        • Drop partition tolerance (CA)
        • Drop availability (CP)
        • Drop consistency (AP)
      • “Drop” here is usually not meant as binary, but rather tunable
    • Non-relational databases
      • The solution these companies came up with are a family of database for handling web scale:
        • BigTable (developed at Google)
        • Hbase (developed at Yahoo!)
        • Dynamo (developed at Amazon)
        • Cassandra (developed at FaceBook)
        • Voldemort (developed at LinkedIn)
        • & a few more:
          • Riak, Redis, CouchDB, MongoDB, Hypertable
    • Benefits
      • Massively scalable
      • Extremely fast
      • Highly available, decentralized & fault tolerant (no single-point-of-failure)
      • Transparent sharding (consistent hashing)
      • Elasticity
      • Parallel processing
      • Dynamic schema
      • Automatic conflict resolution
    • Consistent hashing
    • Replication
    • Replication – node joining
    • Replication – node leaving
    • Scale-out / elasticity?
      • O(1) Distributed Hashtable
      • Runs on a large number of cheap commodity machines
      • Replication
      • Gossip protocol
      • Transparently handles adding/removing nodes
    • Tunable consistency?
      • Levels of consistency:
        • Strict consistency
        • Read your writes consistency
        • Session consistency
        • Monotonic read consistency
        • Eventual consistency
      • Tunable means: how many replica's to lock on write
        • N, R, W parameters
        • Quorum
    • Dealing with inconsistency
      • Read-repair (when encountering inconsistency)
      • Vector clock conflict resolution
    • Dynamic schema
      • Column families (basically a sparse table)
    • Dynamic schema (contd.)
      • “Supercolumn” is a collection of columns
      • Record can have several “supercolumns”
    • Data processing
      • Map/Reduce: an API exposed by non-relational databases to process data
        • A functional programming pattern for parallelizing work
        • Brings the workers to the data – excellent fit for non-relational databases
        • Minimizes the programming to 2 simple functions (map & reduce)
        • Example: count appearances of a word in a giant table of large texts
    • Map/Reduce (contd.)
    • Storage
    • Cost
      • Allows sacrificing consistency (ACID) - at certain circumstances (but can deal with it)
      • Non-standard new API model
      • Non-standard new Schema model
      • New knowledge required to tune/optimize
      • Less mature
    • API model
      • Usually, similar to Key-Value map:
        • Get(key)
        • Put(key, value)
        • Delete(key)
        • Execute(operation, key_list)
      • “value” can be
        • an opaque serialized object
        • a record (list of “columns”: <name, value, timestamp>)
    • Schema model
      • Kind of sparse table
      • No schema
    • Example: Cassandra
      • Features:
        • O(1) DHT
        • Eventual consistency
          • tunable: consistency vs. latency
        • Values are structured, indexed
        • Columns / column families
        • Slicing with predicates (queries)
        • PartitionOrderer
    • Cassandra performance
      • Benchmark against MySQL (50GB)
        • MySQL:
          • 300ms write
          • 350ms read
        • Cassandra:
          • 0.12ms write
          • 15ms read
      • how come writes are so fast?
        • Writes involve no reads/seeks
        • Use any node (closest to you)
    • Cassandra API
    • Cassandra API (contd.)
    • Example: Cassandra (contd.)
      • Java API
        • Simple DAO
        • Simple client
    • Cassandra usage
      • Very high-traffic sites:
        • Facebook
        • Digg
        • Twitter
    • Further information
      • The Dynamo paper:
        • http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
      • Nosql patterns:
        • http://horicky.blogspot.com/2009/11/nosql-patterns.html
      • Nosql conference video's:
        • https://nosqleast.com/2009/
      • Hebrew podcast covering nosql & Cassandra (episodes 56, 57 & more):
        • http://www.reversim.com/
    • Further information (contd.)
      • Ran Tavori's lecture (video + slides):
        • http://prettyprint.me/2010/01/09/introduction-to-nosql-and-cassandra-part-1/
        • http://prettyprint.me/2010/01/20/introduction-to-nosql-and-cassandra-part-2/