Mark	
  Hillick
Email	
  :	
  mark@10gen.com
Twitter	
  :	
  @markofu
MongoDB Introduction
Wednesday 31 October 12
Apologies
Source: http://blog.stackoverflow.com/wp-content/
uploads/jarrod-lair-1.jpg
Source: http://abcnews.go.com/blogs/h...
NoSQL
Wednesday 31 October 12
What?
• A collection of very different products
• Not relational
• No joins
• Currently not using SQL
• Allows much more fl...
10gen & MongoDB
Wednesday 31 October 12
Who? What?
• 10gen created MongoDB in 2007
• Founded by Eliot Horowitz and Dwight Merriman
• Inspired by work on DoubleCli...
When?
• Coding started fall 2007
• First production site March 2008 - businessinsider.com
• Open Source - AGPL
• Version 0...
Terminology
RDBMS MongoDB
Table Collection
Row(s) JSON	
  Document
Index Index
Join Embedding	
  &	
  Linking
Partition Sh...
Data Snippet
Wednesday 31 October 12
Here is a “simple” SQL Model
mysql> select * from book;
+----+----------------------------------------------------------+
...
The Same Data in MongoDB
{
"_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"),
"title" : "Programming in Scala",
"author" : [
{
...
CRUD Operations
Wednesday 31 October 12
Insert
• You should see something like:
“Inserted 1 record(s) in 2ms”
• db.test.insert({name: “mark”, surname:
“hillick”, ...
Find
• db.test.find({name: “mark”})
• You should see something like:
“{ "_id":
ObjectId("509108ff707a9dd29312054e"),
"name"...
Update
• db.name.update({name: 'mark'}, {$set:
{weight: 169}})
• You should see something like:
“Updated 1 existing record...
Delete
• You should see something like:
“Removed 1 record(s) in 0ms”
• db.name.remove({name: “mark”})
Wednesday 31 October...
Installation
Wednesday 31 October 12
Where to Download
• Binaries: http://www.mongodb.org/downloads
• Subscription: http://ww.mongodb.org/mongodb-
subscriber-e...
What now?
• How To Install: http://docs.mongodb.org/manual/
installation/
• Release Notes: http://docs.mongodb.org/manual/...
Other Documentation
• Binaries - http://www.mongodb.org/downloads
• New: http://docs.mongodb.org/manual/
• Tutorials
• Des...
Replication
Wednesday 31 October 12
Types of outage
• Resilience
• Reliability
• Protect against planned & unplanned outages
Wednesday 31 October 12
How MongoDB Replication works
Member	
  1
Member	
  2
Member	
  3
•Set is made up of 2 or more nodes
Wednesday 31 October ...
How MongoDB Replication works
Member	
  1
Member	
  2
PRIMARY
Member	
  3
•Election establishes the PRIMARY
•Data replicat...
How MongoDB Replication works
Member	
  1
Member	
  2
DOWN
Member	
  3
negotiate	
  
new	
  master
•PRIMARY may fail
•Auto...
How MongoDB Replication works
Member	
  1
Member	
  2
DOWN
Member	
  3
PRIMARY
•New PRIMARY elected
•Replication Set re-es...
How MongoDB Replication works
Member	
  1
Member	
  2
RECOVERING
Member	
  3
PRIMARY
•Automatic recovery
Wednesday 31 Octo...
How MongoDB Replication works
Member	
  1
Member	
  2
Member	
  3
PRIMARY
•Replication Set re-established
Wednesday 31 Oct...
Sharding
Wednesday 31 October 12
http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/
theqlikviewblog/Cu...
http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg
Wednesday 31 October 12
MongoDB Scaling - Single Node
write
read
node_a1
Wednesday 31 October 12
Write scaling - add Shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
Wednesday 31 October 12
Write scaling - add Shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
shard3
node_c3
node_b3...
MongoDB Sharding
• Automatic partitioning and management
• Range based
• Convert to sharded system with no downtime
• Full...
How MongoDB Sharding works
>	
  db.posts.save(	
  {age:40}	
  )
-∞   +∞  
-∞   40 41 +∞  
•Data in inserted
•Ranges are sp...
How MongoDB Sharding works
>	
  db.posts.save(	
  {age:40}	
  )
>	
  db.posts.save(	
  {age:50}	
  )
>	
  db.posts.save(	
...
Split Example
Wednesday 31 October 12
{	
  
	
  	
  name:	
  “Jared”,
	
  	
  email:	
  “jsr@10gen.com”,
}
{	
  
	
  	
  name:	
  “Scott”,
	
  	
  email:	
  
“s...
Chunks 1
-∞ +∞
Wednesday 31 October 12
-∞ +∞
dan@10gen.com
jsr@10gen.com
scott@10gen.com
Chunks 2
Wednesday 31 October 12
Chunks 3
-∞ +∞
dan@10gen.com
jsr@10gen.com
scott@10gen.com
Split!
Wednesday 31 October 12
-∞ +∞
dan@10gen.com
jsr@10gen.com
scott@10gen.com
Split!This is a
chunk
This is a
chunk
Chunks 4
Wednesday 31 October 12
-∞ +∞
dan@10gen.com
jsr@10gen.com
scott@10gen.com
Chunks 5
Wednesday 31 October 12
Chunks 6
-∞ +∞
dan@10gen.com
jsr@10gen.com
scott@10gen.com
Split!
Wednesday 31 October 12
Balancing Example
Wednesday 31 October 12
Balancing 1
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
17
21
13
18
22
14
19
23
15
20
24
16
29
33
25
30
34
...
Balancing 2
mongos
balancer
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
21 22 23 24 33...
Balancing 3
mongos
balancer
Move chunk 1 to
Shard 2
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11...
Balancing 4
mongos
balancer
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
6
10
2
7
11
3
8
12
4
21 22 23 24 33 3...
Balancing 4
mongos
balancer
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
6
10
2
7
11
3
8
12
4
21 22 23 24 33 3...
Balancing 5
mongos
balancer
Chunk 1 now lives on
Shard 2
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
16
10
2
...
Drivers
Wednesday 31 October 12
Documentation & Features
• Doc: http://www.mongodb.org/display/DOCS/Drivers
• Official
• C, C++, Java, PHP, Python, Ruby, ...
Operators & Indexes
Wednesday 31 October 12
Operators
> var c = db.test.find({x: 20}).skip(20).limit(10)> c.next()
> c.next()
...
$gt, $lt, $gte, $lte, $ne, $all, $in,...
db.blogs.ensureIndex({author:	
  1})
1	
  =	
  ascending
-­‐1	
  =	
  descending
An	
  index	
  on	
  _id	
  is	
  automat...
Compound Indexes
db.blogs.save({
	
  	
  author:	
  "James",
	
  	
  ts:	
  new	
  Date()
	
  	
  ...
});
db.blogs.ensureI...
db.blogs.save({
	
  	
  title:	
  "My	
  First	
  blog",
	
  	
  comments:	
  [	
  
	
  	
  	
  	
  {author:	
  "James",	
...
db.blogs.save({
	
  	
  loc:	
  {	
  long:	
  40.739037,	
  lat:	
  40.739037	
  }
});
db.blogs.save({
	
  	
  loc:	
  [40...
Security
Wednesday 31 October 12
Documentation & Features
• New: http://docs.mongodb.org/manual/security/
• Linux
• Windows
• AMI
• Authentication
• SSL
• ...
Future
• Role-based Authentication
• Kerberos
• LDAP (external authorisation)
Wednesday 31 October 12
Roadmap
Wednesday 31 October 12
2.2 Features
• Aggregation Framework (no need for map-reduce)
•m/r single thread @ present
• TTL Collections
• Concurrency...
Future
• 2.4
• Security, security, security
• Faster Aggregation
• Hash-based shard keys
• More ops
Wednesday 31 October 12
Get Involved
Wednesday 31 October 12
Where
• Open-Source Community
• Contribute - tell us your thoughts!!!
• Support provided By 10gen & Community on Google
Gr...
Education
Wednesday 31 October 12
https://education.10gen.com/
Learn for Free :)
Wednesday 31 October 12
Meetup Groups
Wednesday 31 October 12
MUGs
• Right Here in Dublin: http://mongodb.meetup.com/
cities/ie/dublin/ :)
• Complete List: http://www.mongodb.org/displ...
Dublin
• Next MeetUp: Thursday, December 6th @ EngineYard
• Poll: http://www.meetup.com/DublinMUG/polls/
Source: http://ww...
Finally
Wednesday 31 October 12
Thanks
Source: http://smallbiztrends.com/wp-content/uploads/
2010/01/question-things.jpg
Wednesday 31 October 12
Upcoming SlideShare
Loading in …5
×

Introduction to MongoDB

1,044 views

Published on

Introduction

Published in: Technology
  • Be the first to comment

Introduction to MongoDB

  1. 1. Mark  Hillick Email  :  mark@10gen.com Twitter  :  @markofu MongoDB Introduction Wednesday 31 October 12
  2. 2. Apologies Source: http://blog.stackoverflow.com/wp-content/ uploads/jarrod-lair-1.jpg Source: http://abcnews.go.com/blogs/headlines/ 2012/10/hurricane-sandy-live-updates/ mark != dev = no prep :( Wednesday 31 October 12
  3. 3. NoSQL Wednesday 31 October 12
  4. 4. What? • A collection of very different products • Not relational • No joins • Currently not using SQL • Allows much more flexible data structures than RDBMS Wednesday 31 October 12
  5. 5. 10gen & MongoDB Wednesday 31 October 12
  6. 6. Who? What? • 10gen created MongoDB in 2007 • Founded by Eliot Horowitz and Dwight Merriman • Inspired by work on DoubleClick, ShopWiki and Panther Express • Over 170 people • Funded by: Sequoia, Flybridge Capital, Union Square Ventures & New Enterprise Associates • Offices in New York (HQ), Palo Alto, London, Dublin and Sydney Wednesday 31 October 12
  7. 7. When? • Coding started fall 2007 • First production site March 2008 - businessinsider.com • Open Source - AGPL • Version 0.8 – first official release February 2009 • Version 1.0 – August 2009 • Version 2.0 – September 2011 • Version 2.2 - September 2012 Wednesday 31 October 12
  8. 8. Terminology RDBMS MongoDB Table Collection Row(s) JSON  Document Index Index Join Embedding  &  Linking Partition Shard Partition  Key Shard  Key Wednesday 31 October 12
  9. 9. Data Snippet Wednesday 31 October 12
  10. 10. Here is a “simple” SQL Model mysql> select * from book; +----+----------------------------------------------------------+ | id | title | +----+----------------------------------------------------------+ | 1 | The Demon-Haunted World: Science as a Candle in the Dark | | 2 | Cosmos | | 3 | Programming in Scala | +----+----------------------------------------------------------+ 3 rows in set (0.00 sec) mysql> select * from bookauthor; +---------+-----------+ | book_id | author_id | +---------+-----------+ | 1 | 1 | | 2 | 1 | | 3 | 2 | | 3 | 3 | | 3 | 4 | +---------+-----------+ 5 rows in set (0.00 sec) mysql> select * from author; +----+-----------+------------+-------------+-------------+---------------+ | id | last_name | first_name | middle_name | nationality | year_of_birth | +----+-----------+------------+-------------+-------------+---------------+ | 1 | Sagan | Carl | Edward | NULL | 1934 | | 2 | Odersky | Martin | NULL | DE | 1958 | | 3 | Spoon | Lex | NULL | NULL | NULL | | 4 | Venners | Bill | NULL | NULL | NULL | +----+-----------+------------+-------------+-------------+---------------+ 4 rows in set (0.00 sec) Wednesday 31 October 12
  11. 11. The Same Data in MongoDB { "_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"), "title" : "Programming in Scala", "author" : [ { "first_name" : "Martin", "last_name" : "Odersky", "nationality" : "DE", "year_of_birth" : 1958 }, { "first_name" : "Lex", "last_name" : "Spoon" }, { "first_name" : "Bill", "last_name" : "Venners" } ] } Wednesday 31 October 12
  12. 12. CRUD Operations Wednesday 31 October 12
  13. 13. Insert • You should see something like: “Inserted 1 record(s) in 2ms” • db.test.insert({name: “mark”, surname: “hillick”, weight: 170}) Wednesday 31 October 12
  14. 14. Find • db.test.find({name: “mark”}) • You should see something like: “{ "_id": ObjectId("509108ff707a9dd29312054e"), "name": "mark", "surname": "mark", "weight": 170 }” Wednesday 31 October 12
  15. 15. Update • db.name.update({name: 'mark'}, {$set: {weight: 169}}) • You should see something like: “Updated 1 existing record(s) in 0ms” Wednesday 31 October 12
  16. 16. Delete • You should see something like: “Removed 1 record(s) in 0ms” • db.name.remove({name: “mark”}) Wednesday 31 October 12
  17. 17. Installation Wednesday 31 October 12
  18. 18. Where to Download • Binaries: http://www.mongodb.org/downloads • Subscription: http://ww.mongodb.org/mongodb- subscriber-edition-download • SSL & SNMP • Source: https://github.com/mongodb/mongo • Packages: RH-type & Ubuntu/Debian Wednesday 31 October 12
  19. 19. What now? • How To Install: http://docs.mongodb.org/manual/ installation/ • Release Notes: http://docs.mongodb.org/manual/ release-notes/ Wednesday 31 October 12
  20. 20. Other Documentation • Binaries - http://www.mongodb.org/downloads • New: http://docs.mongodb.org/manual/ • Tutorials • Descriptions • Configurations • Security (personal favourite) • Old: http://www.mongodb.org/display/DOCS/Home Wednesday 31 October 12
  21. 21. Replication Wednesday 31 October 12
  22. 22. Types of outage • Resilience • Reliability • Protect against planned & unplanned outages Wednesday 31 October 12
  23. 23. How MongoDB Replication works Member  1 Member  2 Member  3 •Set is made up of 2 or more nodes Wednesday 31 October 12
  24. 24. How MongoDB Replication works Member  1 Member  2 PRIMARY Member  3 •Election establishes the PRIMARY •Data replication from PRIMARY to SECONDARY Wednesday 31 October 12
  25. 25. How MongoDB Replication works Member  1 Member  2 DOWN Member  3 negotiate   new  master •PRIMARY may fail •Automatic election of new PRIMARY if majority exists Wednesday 31 October 12
  26. 26. How MongoDB Replication works Member  1 Member  2 DOWN Member  3 PRIMARY •New PRIMARY elected •Replication Set re-established Wednesday 31 October 12
  27. 27. How MongoDB Replication works Member  1 Member  2 RECOVERING Member  3 PRIMARY •Automatic recovery Wednesday 31 October 12
  28. 28. How MongoDB Replication works Member  1 Member  2 Member  3 PRIMARY •Replication Set re-established Wednesday 31 October 12
  29. 29. Sharding Wednesday 31 October 12
  30. 30. http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/ theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg Wednesday 31 October 12
  31. 31. http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg Wednesday 31 October 12
  32. 32. MongoDB Scaling - Single Node write read node_a1 Wednesday 31 October 12
  33. 33. Write scaling - add Shards write read shard1 node_c1 node_b1 node_a1 shard2 node_c2 node_b2 node_a2 Wednesday 31 October 12
  34. 34. Write scaling - add Shards write read shard1 node_c1 node_b1 node_a1 shard2 node_c2 node_b2 node_a2 shard3 node_c3 node_b3 node_a3 Wednesday 31 October 12
  35. 35. MongoDB Sharding • Automatic partitioning and management • Range based • Convert to sharded system with no downtime • Fully consistent Wednesday 31 October 12
  36. 36. How MongoDB Sharding works >  db.posts.save(  {age:40}  ) -∞   +∞   -∞   40 41 +∞   •Data in inserted •Ranges are split into more “chunks” Wednesday 31 October 12
  37. 37. How MongoDB Sharding works >  db.posts.save(  {age:40}  ) >  db.posts.save(  {age:50}  ) >  db.posts.save(  {age:60}  ) -∞   +∞   -∞   40 41 +∞   41 50 51 +∞   61 +∞  51 60 Wednesday 31 October 12
  38. 38. Split Example Wednesday 31 October 12
  39. 39. {      name:  “Jared”,    email:  “jsr@10gen.com”, } {      name:  “Scott”,    email:   “scott@10gen.com”, } {   >  db.runCommand(  {  shardcollection:  “test.users”,                                      key:  {  email:  1  }}  ) Wednesday 31 October 12
  40. 40. Chunks 1 -∞ +∞ Wednesday 31 October 12
  41. 41. -∞ +∞ dan@10gen.com jsr@10gen.com scott@10gen.com Chunks 2 Wednesday 31 October 12
  42. 42. Chunks 3 -∞ +∞ dan@10gen.com jsr@10gen.com scott@10gen.com Split! Wednesday 31 October 12
  43. 43. -∞ +∞ dan@10gen.com jsr@10gen.com scott@10gen.com Split!This is a chunk This is a chunk Chunks 4 Wednesday 31 October 12
  44. 44. -∞ +∞ dan@10gen.com jsr@10gen.com scott@10gen.com Chunks 5 Wednesday 31 October 12
  45. 45. Chunks 6 -∞ +∞ dan@10gen.com jsr@10gen.com scott@10gen.com Split! Wednesday 31 October 12
  46. 46. Balancing Example Wednesday 31 October 12
  47. 47. Balancing 1 Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 17 21 13 18 22 14 19 23 15 20 24 16 29 33 25 30 34 26 31 35 27 32 36 28 41 45 37 42 46 38 43 47 39 44 48 40 mongos balancer config config config Chunks! Wednesday 31 October 12
  48. 48. Balancing 2 mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 ImbalanceImbalance Wednesday 31 October 12
  49. 49. Balancing 3 mongos balancer Move chunk 1 to Shard 2 config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 Wednesday 31 October 12
  50. 50. Balancing 4 mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 1 Wednesday 31 October 12
  51. 51. Balancing 4 mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 1 Wednesday 31 October 12
  52. 52. Balancing 5 mongos balancer Chunk 1 now lives on Shard 2 config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 16 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 Wednesday 31 October 12
  53. 53. Drivers Wednesday 31 October 12
  54. 54. Documentation & Features • Doc: http://www.mongodb.org/display/DOCS/Drivers • Official • C, C++, Java, PHP, Python, Ruby, Scala & more • Community Wednesday 31 October 12
  55. 55. Operators & Indexes Wednesday 31 October 12
  56. 56. Operators > var c = db.test.find({x: 20}).skip(20).limit(10)> c.next() > c.next() ... $gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $or, $not, $mod, $size, $exists, $type, $elemMatch Wednesday 31 October 12
  57. 57. db.blogs.ensureIndex({author:  1}) 1  =  ascending -­‐1  =  descending An  index  on  _id  is  automatic. For  more  use  ensureIndex: Creating Indexes Wednesday 31 October 12
  58. 58. Compound Indexes db.blogs.save({    author:  "James",    ts:  new  Date()    ... }); db.blogs.ensureIndex({author:  1,  ts:  -­‐1}) Wednesday 31 October 12
  59. 59. db.blogs.save({    title:  "My  First  blog",    comments:  [          {author:  "James",  ts  :  new  Date()}  ] }); db.blogs.ensureIndex({"comments.author":  1}) db.blogs.find({"comments.author":  "James"}) Indexing Embedded Arrays Wednesday 31 October 12
  60. 60. db.blogs.save({    loc:  {  long:  40.739037,  lat:  40.739037  } }); db.blogs.save({    loc:  [40.739037,  40.739037] }); db.blogs.ensureIndex({"loc":  "2d"}) Geospatial • Geo hash stored in B-Tree • First two values indexed Wednesday 31 October 12
  61. 61. Security Wednesday 31 October 12
  62. 62. Documentation & Features • New: http://docs.mongodb.org/manual/security/ • Linux • Windows • AMI • Authentication • SSL • NoSQL solutions behind traditional RDBMs Wednesday 31 October 12
  63. 63. Future • Role-based Authentication • Kerberos • LDAP (external authorisation) Wednesday 31 October 12
  64. 64. Roadmap Wednesday 31 October 12
  65. 65. 2.2 Features • Aggregation Framework (no need for map-reduce) •m/r single thread @ present • TTL Collections • Concurrency Improvements • Tag Aware Sharding • Fully Supported Read Semantics • Improved Authentication Wednesday 31 October 12
  66. 66. Future • 2.4 • Security, security, security • Faster Aggregation • Hash-based shard keys • More ops Wednesday 31 October 12
  67. 67. Get Involved Wednesday 31 October 12
  68. 68. Where • Open-Source Community • Contribute - tell us your thoughts!!! • Support provided By 10gen & Community on Google Groups [ groups.google.com/group/mongodb-user] • jira.mongodb.org: features, commercial support, security, bugs, drivers etc Wednesday 31 October 12
  69. 69. Education Wednesday 31 October 12
  70. 70. https://education.10gen.com/ Learn for Free :) Wednesday 31 October 12
  71. 71. Meetup Groups Wednesday 31 October 12
  72. 72. MUGs • Right Here in Dublin: http://mongodb.meetup.com/ cities/ie/dublin/ :) • Complete List: http://www.mongodb.org/display/ DOCS/MongoDB+User+Groups+(MUGs) Wednesday 31 October 12
  73. 73. Dublin • Next MeetUp: Thursday, December 6th @ EngineYard • Poll: http://www.meetup.com/DublinMUG/polls/ Source: http://www.feedhenry.com Source: http://www.iamstaggered.com/wp-content/ uploads/2011/03/dublin.jpg Wednesday 31 October 12
  74. 74. Finally Wednesday 31 October 12
  75. 75. Thanks Source: http://smallbiztrends.com/wp-content/uploads/ 2010/01/question-things.jpg Wednesday 31 October 12

×