Rooster Tech Talk

1,579 views

Published on

A little talk I did internally at SocialVibe, about Rooster: A plugin to replace all of the daemons in our system.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,579
On SlideShare
0
From Embeds
0
Number of Embeds
131
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rooster Tech Talk

  1. 1. Rooster
  2. 2. Rooster • It does the same thing everyday at the same time, whether you are sleeping or not
  3. 3. Rooster • Designed to run periodic tasks, on schedule
  4. 4. History • Previously we used the daemons gem plus the daemon generator plugin. • Headaches: • Each daemon is a separate process. • Not very DRY:
  5. 5. Not DRY • Each daemon has two files: • The actual code that runs as a daemon (e.g. network_point_update_daemon.rb) •
  6. 6. Not DRY • Each daemon has two files: • And a daemon control script (e.g. network_point_update_daemon_ctl) •
  7. 7. Not DRY • The only bit of code that actually does our work: AppUser.daily_amount_grant • The Rails environment is loaded within each daemon process! Not good for memory. • No centralized logging/error handling; poignantly recognized when we wanted exceptions emailed to us. • Management of each daemon process required another entry into monit's config file (eww!):
  8. 8. Back To The Future (of scheduled tasks)
  9. 9. Rooster
  10. 10. Rooster What does it do?
  11. 11. Rooster What does it do? Scratches the itches of our previous configuration
  12. 12. Rooster It’s DRY • One daemon, monitored by monit • One Rails environment, loaded once • Easier management:  TCP server (easily accessed by telnet), rake tasks, more to come... • Centralized and configurable error handling and logging: Rooster::Runner.error_handler = lambda { |e| HoptoadNotifier.notify(e) }
  13. 13. How? • Rooster leverages 3 (optionally 4) excellent open source bits of software:
  14. 14. How? • Rooster leverages 3 (optionally 4) excellent open source bits of software: • EventMachine:  provides event-driven I/O using the Reactor pattern.  It lets you write network clients and servers without handling sockets.
  15. 15. How? • Rooster leverages 3 (optionally 4) excellent open source bits of software: • EventMachine:  provides event-driven I/O using the Reactor pattern.  It lets you write network clients and servers without handling sockets. • Rufus Scheduler:   a Ruby gem for scheduling pieces of code (can leverage EventMachine if available).
  16. 16. How? • Rooster leverages 3 (optionally 4) excellent open source bits of software: • EventMachine:  provides event-driven I/O using the Reactor pattern.  It lets you write network clients and servers without handling sockets. • Rufus Scheduler:   a Ruby gem for scheduling pieces of code (can leverage EventMachine if available). • daemons:  A Ruby gem that provides an easy way to wrap existing ruby code to be run as a daemon, and to be controlled by simple start/stop/restart commands.   (I know we moved away from our previous daemons-based solution, but not all daemons are bad).
  17. 17. How? • Rooster leverages 3 (optionally 4) excellent open source bits of software: • EventMachine:  provides event-driven I/O using the Reactor pattern.  It lets you write network clients and servers without handling sockets. • Rufus Scheduler:   a Ruby gem for scheduling pieces of code (can leverage EventMachine if available). • daemons:  A Ruby gem that provides an easy way to wrap existing ruby code to be run as a daemon and to be controlled by simple start/stop/restart commands.   (I know we moved away from our previous daemons-based solution, but not all daemons are bad). • Chronic (Optional) - A handy gem for natural language date/ time parsing.
  18. 18. How? • These components fit together thusly:
  19. 19. How? • These components fit together thusly: • The rooster daemon is started, and then kicks off the Rooster::Runner.
  20. 20. How? • These components fit together thusly: • The rooster daemon is started, and then kicks off the Rooster::Runner. • Rooster::Runner runs the main EventMachine reactor loop, loads a Rufus::Scheduler, loads (and optionally schedules) each rooster task, and starts the ControlServer.
  21. 21. How? • These components fit together thusly: • The rooster daemon is started, and then kicks off the Rooster::Runner. • Rooster::Runner runs the main EventMachine reactor loop, loads a Rufus::Scheduler, loads (and optionally schedules) each rooster task, and starts the ControlServer. • Rooster::ControlServer is a TCP-based server that listens for Rooster control commands (e.g.  schedule, unschedule, exit, etc.).
  22. 22. How? • These components fit together thusly: • The rooster daemon is started, and then kicks off the Rooster::Runner. • Rooster::Runner runs the main EventMachine reactor loop, loads a Rufus::Scheduler, loads (and optionally schedules) each rooster task, and starts the ControlServer. • Rooster::ControlServer is a TCP-based server that listens for Rooster control commands (e.g.  schedule, unschedule, exit, etc.). • Rooster::ControlClient issues commands to the ControlServer; used mainly as a rake helper.
  23. 23. I kan haz A EXAMPLE Pleez?
  24. 24. Example • I want a task that kills all kittens at 4:20pm every day • > script/generate rooster_task KittenKiller • Generates a new templated task in: RAILS_ROOT/lib/rooster/tasks/kitten_killer_task.rb • rake rooster:launch (and then maybe `rake rooster:start TASK=KittenKillerTask`)
  25. 25. Commands • Tag-based commands are handy for controlling only a subset of available tasks • For example, we have separate rooster tasks running on app1 and app3, and are controlled with those server-specific tags.
  26. 26. Future Goals • Make rooster task scheduling blocks DRYer, especially by abstracting away the ActiveRecord connection pool cleanup. • Refactor Rooster::Runner (prettier code). • Add scripts (e.g. script/rooster daemon:start) • On daemon launch, autostart tasks having a certain tag (or accept a lambda, e.g. launch_if => lamba { |task| task.tags.include?(“app1”) } )
  27. 27. References • http://daemons.rubyforge.org/ • http://github.com/jmettraux/rufus-scheduler/tree/master • http://rubyeventmachine.com/ • http://github.com/findchris/rooster/tree/master
  28. 28. The Rooster Has Spoken
  29. 29. But wait, there’s more!
  30. 30. Single Slide: Create a Rails plugin and push to github • First, go to github.com, signup, add ssh keys, and create a new project • Then...

×