Inside MongoDB: the Internals of an Open-Source Database

46,371 views

Published on

Slides from a talk about MongoDB internals given at Gluecon 2010.

Published in: Technology
3 Comments
81 Likes
Statistics
Notes
  • Njce! Thanks for sharing.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello dear, My name is mariam nasrin, I know that this email will meet you in a good health and also surprisingly but God has his own way of bringing people together. Nice to Meet you I would appreciate if you can reply me back( mariamnasrin2@gmail.com ) So that i can explain you more about me. thank Yours mariam.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • mango db
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
46,371
On SlideShare
0
From Embeds
0
Number of Embeds
6,203
Actions
Shares
0
Downloads
1,018
Comments
3
Likes
81
Embeds 0
No embeds

No notes for slide

Inside MongoDB: the Internals of an Open-Source Database

  1. 1. Mike  Dirolf  -­‐  @mdirolf  -­‐  10gen,  Inc. http://www.mongodb.org
  2. 2. Inside http://www.flickr.com/photos/tmh9/677919415/
  3. 3. a word of warning this  talk  might  be  a  bit  “hard”, but  MongoDB  is  really  easy: http://try.mongodb.org
  4. 4. db.test.insert({hello:  “world”})
  5. 5. _id if  not  specified  drivers  will  add  default: ObjectId("4bface1a2231316e04f3c434") timestamp machine  id process  id counter http://www.mongodb.org/display/DOCS/Object+IDs
  6. 6. BSON Encoding {_id:  ObjectId(XXXXXXXXXXXX),  hello:  “world”} x27x00x00x00x07      _      i      dx00      X          X      X      X      X      X      X      X      X      X      X      Xx02      h      e      l      l      ox00x06x00 x00x00      w      o      r      l      dx00x00 http://bsonspec.org
  7. 7. Insert Message (TCP/IP) message  length request  id response  id op  code  (insert) x68x00x00x00 xXXxXXxXXxXX x00x00x00x00 xd2x07x00x00 reserved collection  name document(s) x00x00x00x00 f  o  o  .  t  e  s  t  x00 BSON  Data http://www.mongodb.org/display/DOCS/Mongo+Wire+Protocol
  8. 8. Data File Allocation $  ls  -­‐sk  /data/db/ }  16384  foo.ns allocated  per   (up  to  2  gigs){ double  in  size    65536  foo.0 131072  foo.1 database  16384  bar.ns              ...
  9. 9. Memory Management
  10. 10. Extent Allocation foo.0 allocated  per  namespace: foo.1 foo.test 00000000000 foo.bar foo.baz 00000000000 00000000000 foo.$freelist 00000000000 0000 preallocated  space foo.2 00000000000 00000000000 ns  details  stored  in  foo.ns 00000000000 00000000000
  11. 11. Record Allocation ... Header  (Size,  Offset,  Next,  Prev) BSON  Data Padding Deleted  Record  (Size,  Offset,  Next) ...
  12. 12. Indexing B-­‐Tree  indexes,  stored  in  own  namespaces >  db.system.namespaces.find() {  "name"  :  "foo.system.indexes"  } {  "name"  :  "foo.test"  } {  "name"  :  "foo.test.$_id_"  } http://www.mongodb.org/display/DOCS/Indexes
  13. 13. db.test.find({hello:  “world”})
  14. 14. Query Language “query  by  example”  plus  $  modifiers: {first_name:  “Mike”,  age:  {$gte:  20,  $lt:  40}} http://www.mongodb.org/display/DOCS/Advanced+Queries
  15. 15. Cursors >  var  c  =  db.test.find({x:  20}).skip(20).limit(10) >  c.next() >  c.next()      ... query first  N  results  +  cursor  id getMore  w/  cursor  id next  N  results  +  cursor  id  or  0 ...
  16. 16. Query Optimizer find({x:  10,  y:  “foo”})    scan terminate    index  on  x    index  on  y remember
  17. 17. db.foo.drop()
  18. 18. Commands drop,  count,  copydb,   findAndModify,  serverStatus,  ... http://www.mongodb.org/display/DOCS/Commands
  19. 19. db.foo.drop(); = db.foo.runCommand({drop:  "foo"}); = db.$cmd.findOne({drop:  "foo"}); = db.$cmd.find({drop:  "foo"}).limit(-­‐1);
  20. 20. Capped Collections preallocated auto  LRI  age-­‐out no  default  _id  index always  in  insertion  order http://www.mongodb.org/display/DOCS/Capped+Collections
  21. 21. Replication Oplog >  use  local switched  to  db  local >  use  foo >  db.oplog.$main.find() switched  to  db  foo {ts:  ...,  op:  "n",  ns:  "",  o:  {}} {ts:  ...,  op:  "n",  ns:  "",  o:  {}} >  db.test.insert({x:  1,  url:   {ts:  ...,  op:  "i",  ns:  "foo.test",   "http://dirolf.com"});  o:  {_id:  ObjectId("..."),          x:  1,          url:  "http://dirolf.com"}} {ts:  ...,  op:  "n",  ns:  "",  o:  {}} >  db.test.update({url:  "http:// {ts:  ...,  op:  "u",  ns:  "foo.test", dirolf.com"},  {$inc:  {x:  1}});  o2:  {_id:  ObjectId("...")},  o:  {$set:  {x:  2}}} http://www.mongodb.org/display/DOCS/Replication
  22. 22. Replication Topology master slave master master slave slave slave slave master master slave master http://www.mongodb.org/display/DOCS/Replication
  23. 23. Auto-Sharding Shards mongod mongod mongod ... Config mongod mongod mongod Servers mongod mongod mongod mongos mongos ... client http://www.mongodb.org/display/DOCS/Sharding
  24. 24. Geohashing (20,  10) (0001  0100,  0000  1010) 0000  0010  0110  0100 maps  close  coordinates  (21,  9)  to  close  hashes: 0000  0010  0110  0011 tricky  part  happens  at  bit-­‐flips  (127  vs  128) http://www.mongodb.org/display/DOCS/Geospatial+Indexing
  25. 25. Download MongoDB http://www.mongodb.org these  slides  are  available  at  http://dirolf.com

×