• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
So you want to liberate your data?
 

So you want to liberate your data?

on

  • 3,364 views

 

Statistics

Views

Total Views
3,364
Views on SlideShare
1,873
Embed Views
1,491

Actions

Likes
1
Downloads
13
Comments
0

7 Embeds 1,491

http://mookid.dk 1462
http://www.medinfo.bg 11
http://guides.library.vcu.edu 7
http://flavors.me 4
http://www.onlydoo.com 3
https://twimg0-a.akamaihd.net 2
http://feeds.feedburner.com 2
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    So you want to liberate your data? So you want to liberate your data? Presentation Transcript

    • d60developing smart software solutions So you want to liberate your data? April 2012
    • Mogens  Heller  Grabe         mhg@d60.dk   @mookid8000   h8p://mookid.dk/oncode  
    • Agenda  •  Data,  queries,  etc.  •  Concurrency  •  AggregaEon  •  Deployment  •  Durability  •  Things  to  be  aware  of  
    • MongoDB  •  Document  database  •  Currently  in  v.  2.0.4  •  Developed  by  10gen  •  Open  source   –  server  is  GNU  AGPL  v3   –  clients  (the  official)  are  Apache  V2  •  Absolutely  free  to  use   –  you  can  get  a  commercial  version  of  the  db  though  –  has   support,  SSL,  and  more  security  features  
    • Conceptual  data  organizaEon   process database collection document process database table row
    • Data  
    • Example  1  •  Install  •  Mongo  Shell  •  Show  database  contents  •  Add  and  show  a  document  
    • Queries  including  several  other  query  operators:  $gt,  $gte,  $lt,  $lte,  $exists,  $all,  etc...  
    • Indexes  
    • Updates  including  several  other  update  modifiers:  $inc,  $set,  $addToSet,  $rename,  etc...  
    • Example  2  •  Import  some  data  •  Query  •  Update  •  Index  •  Query  
    • ACID?  •  Atomic:  Yeah  well,  per  document.  •  Consistent:  Yeah  well,  can  be.  •  Isolated:  Yeah  well,  per  document.  •  Durable:  Yeah  well,  can  be  –  not  default   though....  
    • Concurrency  •  Pushing  it  down  the  stack  
    • Concurrency  •  Preserve  invariants  with  update  precondiEons  
    • Concurrency  •  Use  opEmisEc   locking  when   replacing   document    (and  then  check  whether  n  is  0  or  1...)  
    • Concurrency  •  Use  FindAndModify  to  “check  out”  documents  
    • AggregaEon  •  Map/reduce  
    • AggregaEon  •  Map/reduce   –  Map:   for  each  document:    emit  0  or  more  (key,  value)  tuples   –  Reduce:   given  a  (key,  value[]),    return  1  value  
    • AggregaEon  m  =  function()  {          var  doc  =  this;          doc.appearances.forEach(function(a)  {                  emit(a,  {                          count:  1,                            names:  [doc.firstName  +  “  “  +  doc.lastName]                  });          });  }    r  =  function(key,  values)  {          var  count  =  0;          var  names  =  [];          values.forEach(function(v)  {                  count  +=  v.count;                  names  =  names.concat(v.names);          });          return  {count:  count,  names:  names};  }  
    • Example  3  •  Use  map/reduce  to  collect  informaEon  on   who  appeared  in  each  episode  
    • AggregaEon  •  AggregaEon  framework  (not  available  unEl   2.2)   –  declaraEve  syntax  for  construcEon  of  an   aggregaEon  pipeline  
    • AggregaEon  •  AggregaEon  framework  (not  available  unEl   2.2)  
    • Deployment  •  Several  configuraEons   –  we’ll  check  out  replica  sets  and  sharding  
    • Replica  sets  •  Master-­‐slave  with  automaEc  failover   –  Each  mongod  should  be  started  with  the  -­‐-­‐replset   argument   –  AddiEonal  nodes  added  from  the  shell   –  Make  sure  the  number  of  nodes  is  odd,  possibly   by  adding  an  arbiter  
    • Replica  sets  •  Higher  availability  •  Scale  out  reads  •  Backup  without  interfering  with  the  primary  
    • Sharding  •  Auto-­‐sharding   –  happens  by  user-­‐defined   shard  key   –  can  be  defined  per   collecEon   –  requires  special  nodes:   mongos  (the  load   balancer)  and  a  mongod   that  is  configured  to  be  a   configuraEon  server  
    • Sharding  •  Scale  out  writes  •  LimitaEons:   –  Shard  key  is  immutable   –  All  inserts/updates  must  include  the  shard  key   –  Cannot  enforce  (arbitrary)  uniqueness  across   shards,  only  for  shard  key  
    • Sharding  +  replica  sets  
    • MongoDB’s  durability  story  •  Memory-­‐mapped  files.  •  fsync.  •  Durability  through  replicaEon   –  pre  1.8  •  Durability  through  journaling   –  an  opEon  since  1.8  –  replica  sets  sEll  cool  though   –  default  since  2.0  
    • MongoDB’s  durability  story  •  Inserts  and  updates  are  unsafe  by  default!!   –  only  purpose:  get  awesome  benchmarks   –  bad:  bites  you  in  the  a**  •  Exposed  differently  on  drivers,  but  always   maps  to  db.getLastError()  
    • MongoDB’s  durability  story  •  Conclusion:  It’s  cool  that  you  can   tweak  it  per  operation,  but  it’s   uncool  that  it’s  unsafe.  
    • Things  to  be  aware  of  •  Safe  mode  off  •  32/64  bit  •  Memory-­‐mapped  file  •  Global  write  lock  •  Indexes  should  always  fit  in  RAM  
    • Thanks  for  listening!   mhg@d60.dk   @mookid8000   h8p://mookid.dk/oncode  
    • Image  credits  The  world’s  most  interesEng  man:  h8p://i.qkme.me/3mwy.jpg  Bison:  h8p://www.flickr.com/photos/johan-­‐gril/5632513228/  Tired  Fry:  h8p://cdn.memegenerator.net/instances/400x/18731987.jpg              Thanks  for  lerng  me  borrow  your  awesome  images  –  if  you  ever  meet  me,  I’ll  buy  you  a  beer.  Seriously,  I  will.