MongoUK 2011 - Rplacing RabbitMQ with MongoDB

22,499
-1

Published on

Published in: Technology, Sports
1 Comment
16 Likes
Statistics
Notes
No Downloads
Views
Total Views
22,499
On Slideshare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
182
Comments
1
Likes
16
Embeds 0
No embeds

No notes for slide

MongoUK 2011 - Rplacing RabbitMQ with MongoDB

  1. 1. Replacing RabbitMQ with MongoDB
  2. 2. David MyttonWoop Japan!
  3. 3. Queuing: Useswww.flickr.com/photos/triplexpresso/496995086/
  4. 4. Queuing: Uses• Background processingwww.flickr.com/photos/triplexpresso/496995086/
  5. 5. Queuing: Uses• Background processing• Sending notificationswww.flickr.com/photos/triplexpresso/496995086/
  6. 6. Queuing: Uses• Background processing• Sending notifications• Event streamingwww.flickr.com/photos/triplexpresso/496995086/
  7. 7. Queuing: Features
  8. 8. Queuing: Features• Consumers
  9. 9. Queuing: Features• Consumers• Atomic
  10. 10. Queuing: Features• Consumers• Atomic• Speed
  11. 11. Queuing: Features• Consumers• Atomic• Speed• GC
  12. 12. Queuing: Features•Consumers
  13. 13. Queuing: Features•Consumers MongoDB RabbitMQ Mongo Wire AMQP Protocol
  14. 14. Queuing: Features•Atomicen.wikipedia.org/wiki/State_of_matter
  15. 15. Queuing: Features•Atomic MongoDB RabbitMQ findAndModify consume/acken.wikipedia.org/wiki/State_of_matter
  16. 16. Queuing: Features•Speed
  17. 17. Queuing: Features•GC
  18. 18. Queuing: Features•GC MongoDB RabbitMQ ☹ consume/ack
  19. 19. It’s a little different.
  20. 20. Implementation• Consumers
  21. 21. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } )
  22. 22. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) query: filter (WHERE) { query: { hats: 5 } }
  23. 23. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) sort: selects the first one on multi-match { sort: { added: -1 } }
  24. 24. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) update: modifier object { update: { $set: {inProg: true, start: new Date()} } }
  25. 25. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) remove: true = deletes on return new: true = returns modified object fields: return specific fields upsert: true = create object if !exists()
  26. 26. Implementation• Consumers
  27. 27. Implementation• Consumers def run(self): import time, sched sc = sched.scheduler(time.time, time.sleep) self.poll(sc) sc.run() def poll(self, sc): doStuff() sc.enter(10, 1, self.poll, (sc,))
  28. 28. Implementation• Consumers
  29. 29. Implementation• Consumers from daemon import Daemon class pantalaimon(Daemon): def run(self): # Do stuff pineMarten = pantalaimon(/path/to/pid.pid) pineMarten.start() https://github.com/boxedice/python-daemon
  30. 30. Implementation• Consumers from daemon import Daemon class pantalaimon(Daemon): def run(self): import time, sched sc = sched.scheduler(time.time, time.sleep) self.poll(sc) sc.run() def poll(self, sc): doStuff() sc.enter(10, 1, self.poll, (sc,)) https://github.com/boxedice/python-daemon
  31. 31. Implementation• GC
  32. 32. Implementation• GC now = datetime.datetime.now() difference = datetime.timedelta(seconds=10) timeout = now - difference queue.find({inProg : True, start : {$lte : timeout} })
  33. 33. Fault toleranceReplica sets
  34. 34. Fault toleranceReplica sets Basic automatic failover
  35. 35. Fault toleranceReplica sets Basic automatic failover Writing to n slaves
  36. 36. Fault toleranceReplica sets Basic automatic failover Writing to n slaves Data centre awareness
  37. 37. ScalingCapped Collections
  38. 38. ScalingCapped Collections Extremely fast
  39. 39. ScalingCapped Collections Extremely fast Maintain insertion order (FIFO)
  40. 40. ScalingCapped Collections Extremely fast Maintain insertion order (FIFO) Don’t have to remove()
  41. 41. ScalingSharding
  42. 42. ScalingSharding findAndModify() locks
  43. 43. ScalingSharding findAndModify() locks Scaling writes across machines
  44. 44. ScalingSharding
  45. 45. ScalingSharding Not with capped collections
  46. 46. Problems / Mongo quirks
  47. 47. Problems / Mongo quirksMeta
  48. 48. Problems / Mongo quirksMetaDurability
  49. 49. Problems / Mongo quirksMetaDurabilityGlobal lock
  50. 50. Stick with RabbitMQ?
  51. 51. Stick with RabbitMQ?QoS
  52. 52. Stick with RabbitMQ?QoSAMQP
  53. 53. Stick with RabbitMQ?QoSAMQPThrottling
  54. 54. David Mytton @davidmyttondavid@boxedice.comwww.mongomonitor.comWoop Japan!

×