Dropping ACID with MongoDB
Upcoming SlideShare
Loading in...5
×
 

Dropping ACID with MongoDB

on

  • 8,692 views

http://yapc2010.com/yn2010/talk/2578

http://yapc2010.com/yn2010/talk/2578

Statistics

Views

Total Views
8,692
Views on SlideShare
8,687
Embed Views
5

Actions

Likes
15
Downloads
79
Comments
1

3 Embeds 5

http://twitter.com 2
https://twitter.com 2
http://www.onlydoo.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • MongoDB is fast for especially k/v pair lookupFast inserts - send-and-forgetIn-place updates
  • MongoDB is fast for especially k/v pair lookupFast inserts - send-and-forgetIn-place updates
  • MongoDB is document oriented
  • Read performance

Dropping ACID with MongoDB Dropping ACID with MongoDB Presentation Transcript

  • Who am I?
  • what is
    ?
  • Application
    Perl
    Apache
    Linux
  • Application
    Perl
    Database
    Apache
    Linux
  • Application
    Perl
    Apache
    Windows
  • Application
    Perl
    Apache
    Linux
  • Application
    Perl
    Apache
    Linux
  • Application
    Perl
    Apache
    Linux
  • Application
    The world's most
    popular open source
    database
    Perl
    Apache
    Linux
  • Application
    Perl
    Apache
    Linux
    The world's most
    advanced open source
    database
  • databases
    contain tables
    contain rows
  • schema
    databases
    contain tables
    contain rows
  • schema
    databases
    contain tables
    contain rows
    joins
  • schema
    databases
    contain tables
    contain rows
    joins
  • Application
    Perl
    Apache
    Linux
  • getting started
  • www.mongodb.org
  • cpan -i MongoDB
  • $conn = MongoDB::Connection->new;
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo;
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo;
    NO ADMINISTRATION
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo;
    $table = $db->bar;
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo;
    $table = $db->bar;
  • server
  • server
    databases
  • server
    databases
    contain tables
  • server
    databases
    contain tables
  • server
    databases
    collections
    contain tables
  • server
    databases
    collections
    contain tables
    contain rows
  • server
    databases
    collections
    contain tables
    contain rows
  • server
    databases
    collections
    contain tables
    documents
    contain rows
  • server
    schema
    databases
    collections
    contain tables
    documents
    contain rows
  • server
    schema
    databases
    collections
    contain tables
    documents
    contain rows
    joins
  • server
    databases
    contain collections
    contain documents
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo;
    $collection = $db->bar;
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo;
    $collection = $db->bar;
    NO ADMINISTRATION
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo
    $collection = $db->bar;
    $doc = {"name" => "kristina",
    "contact info" => {
    "twitter" => "@kchodorow",
    "email" => "kristina@10gen.com"
    },
    "friends" => 400232,
    "pic" => $file_str,
    "member since" => DateTime->now};
  • $conn = MongoDB::Connection->new;
    $db = $conn->foo
    $collection = $db->bar;
    $doc = {"name" => "kristina",
    "contact info" => {
    "twitter" => "@kchodorow",
    "email" => "kristina@10gen.com"
    },
    "friends" => 400232,
    "pic" => $file_str,
    "member since" => DateTime->now};
    $collection->insert($doc);
  • mongo is type-sensitive
    and type-rich
    • null
    • boolean
    • integer
    • double
    • string
    • array
    • hash
    • qr/fobar/i
    • $bindata
    • DateTime
    • MongoDB::OID
    • MongoDB::Code
    • MongoDB::MinKey
    • MongoDB::MaxKey
  • $collection->find_one({
    "name" => "kristina"});
  • $collection->find_one({
    "name" => "kristina"});
    $collection->find_one({
    "contact.twitter" => "@kchodorow"});
  • $collection->find_one({
    "name" => "kristina"});
    $collection->find_one({
    "contact.twitter" => "@kchodorow"});
    $collection->find({
    "member since" => {
    '$gt' => $yesterday,
    '$lt' => $now
    }});
  • $collection->find_one({
    "name" => "kristina"});
    $collection->find_one({
    "contact.twitter" => "@kchodorow"});
    $collection->find({
    "member since" => {
    '$gt' => $yesterday,
    '$lt' => $now
    }})->sort({"friends" => 1});
  • $collection->find_one({
    "name" => "kristina"});
    $collection->find_one({
    "contact.twitter" => "@kchodorow"});
    $collection->find({
    "member since" => {
    '$gt' => $yesterday,
    '$lt' => $now
    }})->sort({"friends" => 1})->limit(10);
  • $collection->find_one({
    "name" => "kristina"});
    $collection->find_one({
    "contact.twitter" => "@kchodorow"});
    $collection->find({
    "member since" => {
    '$gt' => $yesterday,
    '$lt' => $now
    }})->sort({"friends" => 1})->limit(10)
    ->skip(100);
  • $collection->find_one({
    "name" => "kristina"});
    $collection->find_one({
    "contact.twitter" => "@kchodorow"});
    $cursor = $collection->find({
    "member since" => {
    '$gt' => $yesterday,
    '$lt' => $now
    }})->sort({"friends" => 1})->limit(10)
    ->skip(100);
  • while (my $doc = $cursor->next) {
    # ...
    }
    my @results = $cursor->all;
  • x < 3
    {"x" => {'$lt' => 3}}
  • $gt
    $lte
    $ne
    $mod
    $exists
    $type
    ...
  • $collection->update(
    {"name" => "kristina"},
  • $collection->update(
    {"name" => "kristina"},
    {'$set' =>
  • $collection->update(
    {"name" => "kristina"},
    {'$set' =>
    {"contact.website" =>
    "snailinaturtleneck.com"
    }});
  • updates are...
    • atomic
    • in-place (when possible)
    • able to create and delete fields
    • able to create documents
  • $collection->update(
    {"url" => "example.com"},
    {'$inc' => {"pageviews" => 1}},
    {"upsert" => 1}
    );
  • $set
    $inc
    $push
    $pop
    $pull
    ...
  • $set
    $inc
    $push
    $pop
    $pull
    ...
  • $set
    $inc
    $push
    $pop
    $pull
    ...
  • $set
    $inc
    $push
    $pop
    $pull
    ...
  • "$gte"
  • "<null>"
  • ""
  • '$gte' or "$gte"
  • Or define your own!
    ":gte"
    "=gte"
    "?gte"
  • Or define your own!
    " gte"
  • Or define your own!
    " gte"
    " ♕ gte"
  • Or define your own!
    " gte"
    " ♕ gte"
    "xgte"
  • features
  • I suddenly have a deep love for stored procedures
  • I suddenly have a deep love for stored procedures
    now with stored procedures!
  • $db->system->js->insert({
    "_id" => "x",
    "value" => 3});
    $db->system->js->insert({
    "_id" => "y",
    "value" => 4});
    $db->eval("return x+y");
  • my $log = <<<LOG
    function(msg, level) {
    var date = "[" + new Date() + "] ";
    varlvl = level ? level+" " : "";
    print(date + lvl + msg);
    }
    LOG;
    $db->system->js->insert({
    "_id" => "log",
    "value" => $log});
  • $db->eval("log('all your base are
    belong to us', 'FATAL');");
  • $db->eval("log('all your base are
    belong to us', 'FATAL');");
    [Fri May 19 18:34:57] FATAL all your
    base are belong to us
  • J
    J
    J
    J
    J
    J
    J
    J
    J
  • $metadata = {
    "_id" => " "
    "date" => DateTime->now
    "downloads" => 0,
    "user" => $userId,
    "filename" => "rickroll.flv"
    }
    J
  • $grid = $db->get_gridfs;
    $grid->put($fh, $metadata);
  • $grid = $db->get_gridfs;
    $grid->put($fh, $metadata);
    $file = $grid->find_one({
    "filename" => qr/rickroll/i
    });
  • $grid = $db->get_gridfs;
    $grid->put($fh, $metadata);
    $file = $grid->find_one({
    "filename" => qr/rickroll/i
    });
    $file->print($fh2);
  • Benefits:
    • Automatic backups
    • Unlimited metadata
    • Millions of files per directory
    • Random access
    • Better disk locality
  • Geospatial Indexes
  • $coll->insert({location => (-40, 78)});
  • $coll->insert({location => (-40, 78)});
    $coll->find({location => {
    '$near' => (-40, 77)}})
  • $coll->insert({location => (-40, 78)});
    $coll->find({location => {
    '$near' => (-40, 77)}})->limit(10)
  • what doesn't it have?
  • transactions
  • are you practicing safe
    data storage?
  • Insert this.
    Okay, got it.
    Phew, my data's safe.
  • Write this to disk
    I know better than he does, I'll just let this sit in a buffer for a while.
    All over it!
  • ?
    I have no idea what you're talking about.
  • trust no one!
    ...trust a bunch of ones. Mostly.
  • prod1.example.com
    prod2.example.com
  • prod1.example.com
    prod2.example.com
  • prod1.example.com
    prod2.example.com
  • prod2.example.com
    prod1.example.com
  • prod1.example.com
    prod2.example.com
  • prod1.example.com
    prod2.example.com
  • prod1.example.com
    ?
    prod2.example.com
  • prod1.example.com
    prod2.example.com
  • ?
    prod1.example.com
    prod2.example.com
  • prod1.example.com
    prod2.example.com
  • prod1.example.com
    prod2.example.com
  • primary
    secondary
    secondary
    passive
  • okay
  • ?
  • make sure two slaves have this
  • all set
  • EXCEPTION
  • EXCEPTION
  • EXCEPTION
  • $conn->w(3);
    $collection->insert({"x" => 1},
    {"safe" => 1});
  • scaling
  • sometimes read-only
  • sometimes read-only
    sometimes inconsistent
  • U - Z
    K - O
    A - E
    F - J
    P - T
  • U - Z
    K - O
    A - E
    F - J
    P - T
  • U - Z
    K - O
    A - E
    F - J
    P - T
  • mongod
    mongod
    mongod
    config
    mongos
  • mongod
    mongod
    mongod
    mongos
    I want posts from April 2009 - June 2009
  • mongod
    mongod
    mongod
    April and May is on #2. June is on #3.
    mongos
  • mongod
    mongod
    mongod
    mongos
  • mongod
    mongod
    mongod
    mongos
  • mongod
    mongod
    mongod
    mongos
    Add this post for
    March 26, 2010.
  • mongod
    mongod
    mongod
    #3 hold June 2009-present
    mongos
  • mongod
    mongod
    mongod
    config
    So, what's going on?
    mongos
  • mongod
    mongod
    mongod
    config
    mongos
  • $conn = MongoDB::Connection->new;
    # ...
  • try.mongodb.org
  • www.mongodb.org
  • thank you!
    @kchodorow
    http://www.snailinaturtleneck.com