Your SlideShare is downloading. ×
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
MongoDB Schema Design -- Inboxes
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

MongoDB Schema Design -- Inboxes

698

Published on

by Jared Rosoff. Dec 2012

by Jared Rosoff. Dec 2012

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
698
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. #MongoSV 2012Schema Design-- Inboxes!Jared RosoffTechnical Director, 10gen@forjared
  • 2. Agenda• Problem overview• Design Options – Fan out on Read – Fan out on Write – Fan out on Write with Bucketing• Conclusions Single Table En
  • 3. Problem Overview
  • 4. Let’s getSocial
  • 5. Sending Messages ?
  • 6. Reading my Inbox ?
  • 7. Design Options
  • 8. 3 Approaches (there aremore)• Fan out on Read• Fan out on Write• Fan out on Write with Bucketing
  • 9. Fan out on read• Generally, not the right approach• 1 document per message sent• Multiple recipients in an array key• Reading an inbox is finding all messages with my own name in the recipient field• Requires scatter-gather on sharded cluster• Then a lot of random IO on a shard to find everything
  • 10. Fan out on Read// Shard on “from”db.shardCollection(”myapp.messages”, { ”from”: 1} )// Make sure we have an index to handle inbox readsdb.messages.ensureIndex( { ”to”: 1, ”sent”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagedb.messages.save(msg)// Read my inboxdb.messages.find({ to: ”Joe” }).sort({ sent: -1 })
  • 11. Fan out on read – SendMessage Send Message Shard 1 Shard 2 Shard 3
  • 12. Fan out on read – Inbox Read Read Inbox Shard 1 Shard 2 Shard 3
  • 13. Fan out on write• Tends to scale better than fan out on read• 1 document per recipient• Reading my inbox is just finding all of the messages with me as the recipient• Can shard on recipient, so inbox reads hit one shard• But still lots of random IO on the shard
  • 14. Fan out on Write// Shard on “recipient” and “sent”db.shardCollection(”myapp.messages”, { ”recipient”: 1, ”sent”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagefor( recipient in msg.to ) { msg.recipient = recipient db.messages.save(msg);}// Read my inboxdb.messages.find({ recipient: ”Joe” }).sort({ sent: -1 })
  • 15. Fan out on write – SendMessage Send Message Shard 1 Shard 2 Shard 3
  • 16. Fan out on write– Read Inbox Read Inbox Shard 1 Shard 2 Shard 3
  • 17. Fan out on write withbucketing• Generally the best approach• Each “inbox” document is an array of messages• Append a message onto “inbox” of recipient• Bucket inbox documents so there’s not too many per document• Can shard on recipient, so inbox reads hit one shard• 1 or 2 documents to read the whole inbox
  • 18. Fan out on Write// Shard on “owner / sequence”db.shardCollection(”myapp.inbox”, { ”owner”: 1, ”sequence”: 1 } )db.shardCollection(”myapp.users”, { ”user_name”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagefor( recipient in msg.to) { sequence = db.users.findAndModify({ query: { user_name: recipient}, update: { $inc: { ‟msg_count: 1 }}, upsert: true, new: true }).msg_count / 50 db.inbox.update({ owner: recipient, sequence: sequence}, { $push: { „messages‟: msg } }, { upsert: true });}// Read my inboxdb.inbox.find({ owner: ”Joe” }).sort({ sequence: -1 }).limit(2)
  • 19. Bucketed fan out on write -Send Send Message Shard 1 Shard 2 Shard 3
  • 20. Bucketed fan out on write -Read Read Inbox Shard 1 Shard 2 Shard 3
  • 21. Discussion
  • 22. Tradeoffs Fan out on Fan out on Bucketed Fan out Read Write on WriteSend Message Best Good WorstPerformance Single shard Shard per recipient Shard per recipient Single write Multiple writes Appends (grows)Read Inbox Worst Good BestPerformance Broadcast all shards Single shard Single shard Random reads Random reads Single readData Size Best Worst Worst Message stored Copy per recipient Copy per recipient once
  • 23. Things to consider• Lots of recipients • Fan out on write might become prohibitive • Consider introducing a “Group”• Very large message size • Multiple copies of messages can be a burden • Consider single copy of message with a “pointer” per inbox• More writes than reads • Fan out on read might be okay
  • 24. Comments – where do theylive?
  • 25. Conclusion
  • 26. Summary• Multiple ways to model status updates• Bucketed fan out on write is typically the better approach• Think about how your model distributes across shards• Think about how much random IO needs to happen on a shard
  • 27. #MongoSVThank YouJared RosoffTechnical Director, 10gen

×