#MongoSV 2012Schema Design-- Inboxes!Jared RosoffTechnical Director, 10gen@forjared
Agenda• Problem overview• Design Options  – Fan out on Read  – Fan out on Write  – Fan out on Write with Bucketing• Conclu...
Problem Overview
Let’s getSocial
Sending Messages               ?
Reading my Inbox                   ?
Design Options
3 Approaches (there aremore)• Fan out on Read• Fan out on Write• Fan out on Write with Bucketing
Fan out on read• Generally, not the right approach• 1 document per message sent• Multiple recipients in an array key• Read...
Fan out on Read// Shard on “from”db.shardCollection(”myapp.messages”, { ”from”: 1} )// Make sure we have an index to handl...
Fan out on read – SendMessage             Send            Message  Shard 1             Shard 2   Shard 3
Fan out on read – Inbox Read            Read            Inbox  Shard 1           Shard 2   Shard 3
Fan out on write• Tends to scale better than fan out on read• 1 document per recipient• Reading my inbox is just finding a...
Fan out on Write// Shard on “recipient” and “sent”db.shardCollection(”myapp.messages”, { ”recipient”: 1, ”sent”: 1 } )msg ...
Fan out on write – SendMessage             Send            Message  Shard 1             Shard 2   Shard 3
Fan out on write– Read Inbox            Read            Inbox  Shard 1           Shard 2   Shard 3
Fan out on write withbucketing• Generally the best approach• Each “inbox” document is an array of messages• Append a messa...
Fan out on Write// Shard on “owner / sequence”db.shardCollection(”myapp.inbox”, { ”owner”: 1, ”sequence”: 1 } )db.shardCol...
Bucketed fan out on write -Send             Send            Message  Shard 1             Shard 2   Shard 3
Bucketed fan out on write -Read            Read            Inbox  Shard 1           Shard 2   Shard 3
Discussion
Tradeoffs                 Fan out on              Fan out on          Bucketed Fan out                   Read             ...
Things to consider•   Lots of recipients     •   Fan out on write might become prohibitive     •   Consider introducing a ...
Comments – where do theylive?
Conclusion
Summary• Multiple ways to model status updates• Bucketed fan out on write is typically the better approach• Think about ho...
#MongoSVThank YouJared RosoffTechnical Director, 10gen
Upcoming SlideShare
Loading in...5
×

MongoDB Schema Design -- Inboxes

731

Published on

by Jared Rosoff. Dec 2012

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
731
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MongoDB Schema Design -- Inboxes

  1. 1. #MongoSV 2012Schema Design-- Inboxes!Jared RosoffTechnical Director, 10gen@forjared
  2. 2. Agenda• Problem overview• Design Options – Fan out on Read – Fan out on Write – Fan out on Write with Bucketing• Conclusions Single Table En
  3. 3. Problem Overview
  4. 4. Let’s getSocial
  5. 5. Sending Messages ?
  6. 6. Reading my Inbox ?
  7. 7. Design Options
  8. 8. 3 Approaches (there aremore)• Fan out on Read• Fan out on Write• Fan out on Write with Bucketing
  9. 9. Fan out on read• Generally, not the right approach• 1 document per message sent• Multiple recipients in an array key• Reading an inbox is finding all messages with my own name in the recipient field• Requires scatter-gather on sharded cluster• Then a lot of random IO on a shard to find everything
  10. 10. Fan out on Read// Shard on “from”db.shardCollection(”myapp.messages”, { ”from”: 1} )// Make sure we have an index to handle inbox readsdb.messages.ensureIndex( { ”to”: 1, ”sent”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagedb.messages.save(msg)// Read my inboxdb.messages.find({ to: ”Joe” }).sort({ sent: -1 })
  11. 11. Fan out on read – SendMessage Send Message Shard 1 Shard 2 Shard 3
  12. 12. Fan out on read – Inbox Read Read Inbox Shard 1 Shard 2 Shard 3
  13. 13. Fan out on write• Tends to scale better than fan out on read• 1 document per recipient• Reading my inbox is just finding all of the messages with me as the recipient• Can shard on recipient, so inbox reads hit one shard• But still lots of random IO on the shard
  14. 14. Fan out on Write// Shard on “recipient” and “sent”db.shardCollection(”myapp.messages”, { ”recipient”: 1, ”sent”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagefor( recipient in msg.to ) { msg.recipient = recipient db.messages.save(msg);}// Read my inboxdb.messages.find({ recipient: ”Joe” }).sort({ sent: -1 })
  15. 15. Fan out on write – SendMessage Send Message Shard 1 Shard 2 Shard 3
  16. 16. Fan out on write– Read Inbox Read Inbox Shard 1 Shard 2 Shard 3
  17. 17. Fan out on write withbucketing• Generally the best approach• Each “inbox” document is an array of messages• Append a message onto “inbox” of recipient• Bucket inbox documents so there’s not too many per document• Can shard on recipient, so inbox reads hit one shard• 1 or 2 documents to read the whole inbox
  18. 18. Fan out on Write// Shard on “owner / sequence”db.shardCollection(”myapp.inbox”, { ”owner”: 1, ”sequence”: 1 } )db.shardCollection(”myapp.users”, { ”user_name”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagefor( recipient in msg.to) { sequence = db.users.findAndModify({ query: { user_name: recipient}, update: { $inc: { ‟msg_count: 1 }}, upsert: true, new: true }).msg_count / 50 db.inbox.update({ owner: recipient, sequence: sequence}, { $push: { „messages‟: msg } }, { upsert: true });}// Read my inboxdb.inbox.find({ owner: ”Joe” }).sort({ sequence: -1 }).limit(2)
  19. 19. Bucketed fan out on write -Send Send Message Shard 1 Shard 2 Shard 3
  20. 20. Bucketed fan out on write -Read Read Inbox Shard 1 Shard 2 Shard 3
  21. 21. Discussion
  22. 22. Tradeoffs Fan out on Fan out on Bucketed Fan out Read Write on WriteSend Message Best Good WorstPerformance Single shard Shard per recipient Shard per recipient Single write Multiple writes Appends (grows)Read Inbox Worst Good BestPerformance Broadcast all shards Single shard Single shard Random reads Random reads Single readData Size Best Worst Worst Message stored Copy per recipient Copy per recipient once
  23. 23. Things to consider• Lots of recipients • Fan out on write might become prohibitive • Consider introducing a “Group”• Very large message size • Multiple copies of messages can be a burden • Consider single copy of message with a “pointer” per inbox• More writes than reads • Fan out on read might be okay
  24. 24. Comments – where do theylive?
  25. 25. Conclusion
  26. 26. Summary• Multiple ways to model status updates• Bucketed fan out on write is typically the better approach• Think about how your model distributes across shards• Think about how much random IO needs to happen on a shard
  27. 27. #MongoSVThank YouJared RosoffTechnical Director, 10gen
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×