Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MongoUK 2011 - Rplacing RabbitMQ with MongoDB

24,030 views

Published on

Published in: Technology, Sports

MongoUK 2011 - Rplacing RabbitMQ with MongoDB

  1. 1. Replacing RabbitMQ with MongoDB
  2. 2. David MyttonWoop Japan!
  3. 3. Queuing: Useswww.flickr.com/photos/triplexpresso/496995086/
  4. 4. Queuing: Uses• Background processingwww.flickr.com/photos/triplexpresso/496995086/
  5. 5. Queuing: Uses• Background processing• Sending notificationswww.flickr.com/photos/triplexpresso/496995086/
  6. 6. Queuing: Uses• Background processing• Sending notifications• Event streamingwww.flickr.com/photos/triplexpresso/496995086/
  7. 7. Queuing: Features
  8. 8. Queuing: Features• Consumers
  9. 9. Queuing: Features• Consumers• Atomic
  10. 10. Queuing: Features• Consumers• Atomic• Speed
  11. 11. Queuing: Features• Consumers• Atomic• Speed• GC
  12. 12. Queuing: Features•Consumers
  13. 13. Queuing: Features•Consumers MongoDB RabbitMQ Mongo Wire AMQP Protocol
  14. 14. Queuing: Features•Atomicen.wikipedia.org/wiki/State_of_matter
  15. 15. Queuing: Features•Atomic MongoDB RabbitMQ findAndModify consume/acken.wikipedia.org/wiki/State_of_matter
  16. 16. Queuing: Features•Speed
  17. 17. Queuing: Features•GC
  18. 18. Queuing: Features•GC MongoDB RabbitMQ ☹ consume/ack
  19. 19. It’s a little different.
  20. 20. Implementation• Consumers
  21. 21. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } )
  22. 22. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) query: filter (WHERE) { query: { hats: 5 } }
  23. 23. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) sort: selects the first one on multi-match { sort: { added: -1 } }
  24. 24. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) update: modifier object { update: { $set: {inProg: true, start: new Date()} } }
  25. 25. Implementation• Consumers db.runCommand( { findAndModify : <collection>, <options> } ) remove: true = deletes on return new: true = returns modified object fields: return specific fields upsert: true = create object if !exists()
  26. 26. Implementation• Consumers
  27. 27. Implementation• Consumers def run(self): import time, sched sc = sched.scheduler(time.time, time.sleep) self.poll(sc) sc.run() def poll(self, sc): doStuff() sc.enter(10, 1, self.poll, (sc,))
  28. 28. Implementation• Consumers
  29. 29. Implementation• Consumers from daemon import Daemon class pantalaimon(Daemon): def run(self): # Do stuff pineMarten = pantalaimon(/path/to/pid.pid) pineMarten.start() https://github.com/boxedice/python-daemon
  30. 30. Implementation• Consumers from daemon import Daemon class pantalaimon(Daemon): def run(self): import time, sched sc = sched.scheduler(time.time, time.sleep) self.poll(sc) sc.run() def poll(self, sc): doStuff() sc.enter(10, 1, self.poll, (sc,)) https://github.com/boxedice/python-daemon
  31. 31. Implementation• GC
  32. 32. Implementation• GC now = datetime.datetime.now() difference = datetime.timedelta(seconds=10) timeout = now - difference queue.find({inProg : True, start : {$lte : timeout} })
  33. 33. Fault toleranceReplica sets
  34. 34. Fault toleranceReplica sets Basic automatic failover
  35. 35. Fault toleranceReplica sets Basic automatic failover Writing to n slaves
  36. 36. Fault toleranceReplica sets Basic automatic failover Writing to n slaves Data centre awareness
  37. 37. ScalingCapped Collections
  38. 38. ScalingCapped Collections Extremely fast
  39. 39. ScalingCapped Collections Extremely fast Maintain insertion order (FIFO)
  40. 40. ScalingCapped Collections Extremely fast Maintain insertion order (FIFO) Don’t have to remove()
  41. 41. ScalingSharding
  42. 42. ScalingSharding findAndModify() locks
  43. 43. ScalingSharding findAndModify() locks Scaling writes across machines
  44. 44. ScalingSharding
  45. 45. ScalingSharding Not with capped collections
  46. 46. Problems / Mongo quirks
  47. 47. Problems / Mongo quirksMeta
  48. 48. Problems / Mongo quirksMetaDurability
  49. 49. Problems / Mongo quirksMetaDurabilityGlobal lock
  50. 50. Stick with RabbitMQ?
  51. 51. Stick with RabbitMQ?QoS
  52. 52. Stick with RabbitMQ?QoSAMQP
  53. 53. Stick with RabbitMQ?QoSAMQPThrottling
  54. 54. David Mytton @davidmyttondavid@boxedice.comwww.mongomonitor.comWoop Japan!

×