Your SlideShare is downloading. ×
  • Like
MongoDB Schema Design -- Inboxes
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

MongoDB Schema Design -- Inboxes

  • 641 views
Published

by Jared Rosoff. Dec 2012

by Jared Rosoff. Dec 2012

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
641
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
9
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. #MongoSV 2012Schema Design-- Inboxes!Jared RosoffTechnical Director, 10gen@forjared
  • 2. Agenda• Problem overview• Design Options – Fan out on Read – Fan out on Write – Fan out on Write with Bucketing• Conclusions Single Table En
  • 3. Problem Overview
  • 4. Let’s getSocial
  • 5. Sending Messages ?
  • 6. Reading my Inbox ?
  • 7. Design Options
  • 8. 3 Approaches (there aremore)• Fan out on Read• Fan out on Write• Fan out on Write with Bucketing
  • 9. Fan out on read• Generally, not the right approach• 1 document per message sent• Multiple recipients in an array key• Reading an inbox is finding all messages with my own name in the recipient field• Requires scatter-gather on sharded cluster• Then a lot of random IO on a shard to find everything
  • 10. Fan out on Read// Shard on “from”db.shardCollection(”myapp.messages”, { ”from”: 1} )// Make sure we have an index to handle inbox readsdb.messages.ensureIndex( { ”to”: 1, ”sent”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagedb.messages.save(msg)// Read my inboxdb.messages.find({ to: ”Joe” }).sort({ sent: -1 })
  • 11. Fan out on read – SendMessage Send Message Shard 1 Shard 2 Shard 3
  • 12. Fan out on read – Inbox Read Read Inbox Shard 1 Shard 2 Shard 3
  • 13. Fan out on write• Tends to scale better than fan out on read• 1 document per recipient• Reading my inbox is just finding all of the messages with me as the recipient• Can shard on recipient, so inbox reads hit one shard• But still lots of random IO on the shard
  • 14. Fan out on Write// Shard on “recipient” and “sent”db.shardCollection(”myapp.messages”, { ”recipient”: 1, ”sent”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagefor( recipient in msg.to ) { msg.recipient = recipient db.messages.save(msg);}// Read my inboxdb.messages.find({ recipient: ”Joe” }).sort({ sent: -1 })
  • 15. Fan out on write – SendMessage Send Message Shard 1 Shard 2 Shard 3
  • 16. Fan out on write– Read Inbox Read Inbox Shard 1 Shard 2 Shard 3
  • 17. Fan out on write withbucketing• Generally the best approach• Each “inbox” document is an array of messages• Append a message onto “inbox” of recipient• Bucket inbox documents so there’s not too many per document• Can shard on recipient, so inbox reads hit one shard• 1 or 2 documents to read the whole inbox
  • 18. Fan out on Write// Shard on “owner / sequence”db.shardCollection(”myapp.inbox”, { ”owner”: 1, ”sequence”: 1 } )db.shardCollection(”myapp.users”, { ”user_name”: 1 } )msg = { from: "Joe”, to: [ ”Bob”, “Jane” ], sent: new Date(), message: ”Hi!”,}// Send a messagefor( recipient in msg.to) { sequence = db.users.findAndModify({ query: { user_name: recipient}, update: { $inc: { ‟msg_count: 1 }}, upsert: true, new: true }).msg_count / 50 db.inbox.update({ owner: recipient, sequence: sequence}, { $push: { „messages‟: msg } }, { upsert: true });}// Read my inboxdb.inbox.find({ owner: ”Joe” }).sort({ sequence: -1 }).limit(2)
  • 19. Bucketed fan out on write -Send Send Message Shard 1 Shard 2 Shard 3
  • 20. Bucketed fan out on write -Read Read Inbox Shard 1 Shard 2 Shard 3
  • 21. Discussion
  • 22. Tradeoffs Fan out on Fan out on Bucketed Fan out Read Write on WriteSend Message Best Good WorstPerformance Single shard Shard per recipient Shard per recipient Single write Multiple writes Appends (grows)Read Inbox Worst Good BestPerformance Broadcast all shards Single shard Single shard Random reads Random reads Single readData Size Best Worst Worst Message stored Copy per recipient Copy per recipient once
  • 23. Things to consider• Lots of recipients • Fan out on write might become prohibitive • Consider introducing a “Group”• Very large message size • Multiple copies of messages can be a burden • Consider single copy of message with a “pointer” per inbox• More writes than reads • Fan out on read might be okay
  • 24. Comments – where do theylive?
  • 25. Conclusion
  • 26. Summary• Multiple ways to model status updates• Bucketed fan out on write is typically the better approach• Think about how your model distributes across shards• Think about how much random IO needs to happen on a shard
  • 27. #MongoSVThank YouJared RosoffTechnical Director, 10gen