Your SlideShare is downloading. ×
Inside MongoDB: the Internals of an Open-Source Database
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Inside MongoDB: the Internals of an Open-Source Database

28,532
views

Published on

Slides from a talk about MongoDB internals given at Gluecon 2010.

Slides from a talk about MongoDB internals given at Gluecon 2010.

Published in: Technology

1 Comment
63 Likes
Statistics
Notes
No Downloads
Views
Total Views
28,532
On Slideshare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
824
Comments
1
Likes
63
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Mike  Dirolf  -­‐  @mdirolf  -­‐  10gen,  Inc. http://www.mongodb.org
  • 2. Inside http://www.flickr.com/photos/tmh9/677919415/
  • 3. a word of warning this  talk  might  be  a  bit  “hard”, but  MongoDB  is  really  easy: http://try.mongodb.org
  • 4. db.test.insert({hello:  “world”})
  • 5. _id if  not  specified  drivers  will  add  default: ObjectId("4bface1a2231316e04f3c434") timestamp machine  id process  id counter http://www.mongodb.org/display/DOCS/Object+IDs
  • 6. BSON Encoding {_id:  ObjectId(XXXXXXXXXXXX),  hello:  “world”} x27x00x00x00x07      _      i      dx00      X          X      X      X      X      X      X      X      X      X      X      Xx02      h      e      l      l      ox00x06x00 x00x00      w      o      r      l      dx00x00 http://bsonspec.org
  • 7. Insert Message (TCP/IP) message  length request  id response  id op  code  (insert) x68x00x00x00 xXXxXXxXXxXX x00x00x00x00 xd2x07x00x00 reserved collection  name document(s) x00x00x00x00 f  o  o  .  t  e  s  t  x00 BSON  Data http://www.mongodb.org/display/DOCS/Mongo+Wire+Protocol
  • 8. Data File Allocation $  ls  -­‐sk  /data/db/ }  16384  foo.ns allocated  per   (up  to  2  gigs){ double  in  size    65536  foo.0 131072  foo.1 database  16384  bar.ns              ...
  • 9. Memory Management
  • 10. Extent Allocation foo.0 allocated  per  namespace: foo.1 foo.test 00000000000 foo.bar foo.baz 00000000000 00000000000 foo.$freelist 00000000000 0000 preallocated  space foo.2 00000000000 00000000000 ns  details  stored  in  foo.ns 00000000000 00000000000
  • 11. Record Allocation ... Header  (Size,  Offset,  Next,  Prev) BSON  Data Padding Deleted  Record  (Size,  Offset,  Next) ...
  • 12. Indexing B-­‐Tree  indexes,  stored  in  own  namespaces >  db.system.namespaces.find() {  "name"  :  "foo.system.indexes"  } {  "name"  :  "foo.test"  } {  "name"  :  "foo.test.$_id_"  } http://www.mongodb.org/display/DOCS/Indexes
  • 13. db.test.find({hello:  “world”})
  • 14. Query Language “query  by  example”  plus  $  modifiers: {first_name:  “Mike”,  age:  {$gte:  20,  $lt:  40}} http://www.mongodb.org/display/DOCS/Advanced+Queries
  • 15. Cursors >  var  c  =  db.test.find({x:  20}).skip(20).limit(10) >  c.next() >  c.next()      ... query first  N  results  +  cursor  id getMore  w/  cursor  id next  N  results  +  cursor  id  or  0 ...
  • 16. Query Optimizer find({x:  10,  y:  “foo”})    scan terminate    index  on  x    index  on  y remember
  • 17. db.foo.drop()
  • 18. Commands drop,  count,  copydb,   findAndModify,  serverStatus,  ... http://www.mongodb.org/display/DOCS/Commands
  • 19. db.foo.drop(); = db.foo.runCommand({drop:  "foo"}); = db.$cmd.findOne({drop:  "foo"}); = db.$cmd.find({drop:  "foo"}).limit(-­‐1);
  • 20. Capped Collections preallocated auto  LRI  age-­‐out no  default  _id  index always  in  insertion  order http://www.mongodb.org/display/DOCS/Capped+Collections
  • 21. Replication Oplog >  use  local switched  to  db  local >  use  foo >  db.oplog.$main.find() switched  to  db  foo {ts:  ...,  op:  "n",  ns:  "",  o:  {}} {ts:  ...,  op:  "n",  ns:  "",  o:  {}} >  db.test.insert({x:  1,  url:   {ts:  ...,  op:  "i",  ns:  "foo.test",   "http://dirolf.com"});  o:  {_id:  ObjectId("..."),          x:  1,          url:  "http://dirolf.com"}} {ts:  ...,  op:  "n",  ns:  "",  o:  {}} >  db.test.update({url:  "http:// {ts:  ...,  op:  "u",  ns:  "foo.test", dirolf.com"},  {$inc:  {x:  1}});  o2:  {_id:  ObjectId("...")},  o:  {$set:  {x:  2}}} http://www.mongodb.org/display/DOCS/Replication
  • 22. Replication Topology master slave master master slave slave slave slave master master slave master http://www.mongodb.org/display/DOCS/Replication
  • 23. Auto-Sharding Shards mongod mongod mongod ... Config mongod mongod mongod Servers mongod mongod mongod mongos mongos ... client http://www.mongodb.org/display/DOCS/Sharding
  • 24. Geohashing (20,  10) (0001  0100,  0000  1010) 0000  0010  0110  0100 maps  close  coordinates  (21,  9)  to  close  hashes: 0000  0010  0110  0011 tricky  part  happens  at  bit-­‐flips  (127  vs  128) http://www.mongodb.org/display/DOCS/Geospatial+Indexing
  • 25. Download MongoDB http://www.mongodb.org these  slides  are  available  at  http://dirolf.com

×