Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Background Processing with Nanite

4,350 views

Published on

A short presentation I gave to my team regarding Nanite and the benefits of using it for background processing.

  • Be the first to comment

Background Processing with Nanite

  1. 1. Background Processing with Nanite By Jennifer Davis
  2. 2. Overview
  3. 3. Overview
  4. 4. Overview • Nanite
  5. 5. Overview • Nanite • RabbitMQ/AMQP
  6. 6. Overview • Nanite • RabbitMQ/AMQP • Code Example
  7. 7. Overview • Nanite • RabbitMQ/AMQP • Code Example • Questions
  8. 8. Overview
  9. 9. Background Processing
  10. 10. Background Processing
  11. 11. Background Processing Q. Don’t we already do background processing with BackgrounDRb?
  12. 12. Background Processing Q. Don’t we already do background processing with BackgrounDRb? A. Yes, but it is not scalable for our needs.
  13. 13. Background Processing
  14. 14. Background Processing
  15. 15. Background Processing Q. Why Nanite?
  16. 16. Background Processing Q. Why Nanite? A. Although other methods of background processing exist, Nanite is more suited to our needs.
  17. 17. Background Processing
  18. 18. Nanite
  19. 19. Nanite
  20. 20. Nanite “A Self Assembling Cluster of Ruby Daemons” - Ezra Zygmuntowicz
  21. 21. Nanite
  22. 22. Nanite
  23. 23. Nanite Composed of Mappers and Agents
  24. 24. Nanite Composed of Mappers and Agents Thin Mapper
  25. 25. Nanite Composed of Mappers and Agents Thin Mapper Shell Mapper
  26. 26. Nanite Composed of Mappers and Agents Thin A Mapper M Message Queue Q (RabbitMQ) P Shell Mapper
  27. 27. Nanite Composed of Mappers and Agents Agent Thin Actor A Mapper Actor M Message Actor Queue Q (RabbitMQ) P Shell Mapper
  28. 28. Nanite Composed of Mappers and Agents Agent Thin Actor A Mapper Actor M Message Actor Queue Q (RabbitMQ) P Agent Shell Actor Mapper Actor Actor
  29. 29. Nanite Composed of Mappers and Agents Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
  30. 30. Nanite Composed of Mappers and Agents Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
  31. 31. Nanite
  32. 32. Nanite Mappers
  33. 33. Nanite Mappers • Can have one or many
  34. 34. Nanite Mappers • Can have one or many • Run within Rails or the shell
  35. 35. Nanite Mappers • Can have one or many • Run within Rails or the shell • Subscribed to the Mapper Exchange
  36. 36. Nanite Mappers • Can have one or many • Run within Rails or the shell • Subscribed to the Mapper Exchange • Adds/Removes agents based on agent availability
  37. 37. Nanite
  38. 38. Nanite
  39. 39. Nanite Agents
  40. 40. Nanite Agents • Can have one or many
  41. 41. Nanite Agents • Can have one or many • Ping the Mapper Exchange every @ping_time seconds
  42. 42. Nanite Agents • Can have one or many • Ping the Mapper Exchange every @ping_time seconds • A single agent can have many actor classes
  43. 43. Nanite
  44. 44. Nanite
  45. 45. Nanite Selectors for load balancing
  46. 46. Nanite Selectors for load balancing • :least_loaded
  47. 47. Nanite Selectors for load balancing • :least_loaded • :rr
  48. 48. Nanite Selectors for load balancing • :least_loaded • :rr • :random
  49. 49. Nanite Selectors for load balancing • :least_loaded • :rr • :random • :all
  50. 50. Nanite
  51. 51. Nanite Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
  52. 52. Nanite Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
  53. 53. Nanite Ping back status Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
  54. 54. Nanite Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
  55. 55. Nanite
  56. 56. Nanite Nanite requires
  57. 57. Nanite Nanite requires • Erlang
  58. 58. Nanite Nanite requires • Erlang • Nanite ruby gem
  59. 59. Nanite Nanite requires • Erlang • Nanite ruby gem • EventMachine
  60. 60. Nanite Nanite requires • Erlang • Nanite ruby gem • EventMachine • AMQP
  61. 61. Nanite Nanite requires • Erlang • Nanite ruby gem • EventMachine • AMQP • RabbitMQ
  62. 62. Nanite
  63. 63. AMQP
  64. 64. AMQP
  65. 65. AMQP Advanced Messaging Queue Protocol
  66. 66. AMQP Advanced Messaging Queue Protocol • An open Internet Protocol for Business Messaging
  67. 67. AMQP Advanced Messaging Queue Protocol • An open Internet Protocol for Business Messaging • Provides a standard for messaging middleware
  68. 68. AMQP
  69. 69. RabbitMQ
  70. 70. RabbitMQ
  71. 71. RabbitMQ • An AMQP Messaging Broker
  72. 72. RabbitMQ • An AMQP Messaging Broker • Written in Erlang
  73. 73. RabbitMQ • An AMQP Messaging Broker • Written in Erlang • Uses the OTP (Open Telecom Platform) system
  74. 74. RabbitMQ • An AMQP Messaging Broker • Written in Erlang • Uses the OTP (Open Telecom Platform) system • Is stable - Achieved 99.9999999% availability
  75. 75. RabbitMQ
  76. 76. Code Example
  77. 77. Review
  78. 78. Review • Why background processing is important to us • What Nanite is and how to use it • A little background on RabbitMQ
  79. 79. Review
  80. 80. Review • Why background processing is important to us • What Nanite is and how to use it • A little background on RabbitMQ
  81. 81. Resources
  82. 82. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/
  83. 83. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/ • George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/ videos/nanite/
  84. 84. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/ • George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/ videos/nanite/ • RabbitMQ - Open Source Enterprise Messaging at http:// www.rabbitmq.com/
  85. 85. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/ • George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/ videos/nanite/ • RabbitMQ - Open Source Enterprise Messaging at http:// www.rabbitmq.com/ • AMQP - Advanced Message Queuing Protocol at http://jira.amqp.org/ confluence/display/AMQP/Advanced+Message+Queuing+Protocol
  86. 86. Resources
  87. 87. Questions?

×