The document discusses elements of monitored processes like PID files, their long-running nature, and daemonization. It provides examples of using PID files to track processes and ensure only one instance runs. It demonstrates managing long-running processes and using God to monitor and restart them if they crash or exceed resource thresholds. While God provides an easy way to monitor processes, its Ruby DSL is unnatural and it can be a moderately expensive process itself.

1. COUNTING ON GOD
Can we?

2. GOD’S CHILDREN

3. ELEMENTS OF
MONITORED PROCESSES

4. ELEMENTS OF
MONITORED PROCESSES
• PID files

5. ELEMENTS OF
MONITORED PROCESSES
• PID files
• Long running nature (decoupled start/stop)

6. ELEMENTS OF
MONITORED PROCESSES
• PID files
• Long running nature (decoupled start/stop)
• Daemonization

7. PID FILES

8. PID FILE BASICS
module PIDFile
module_function
def create(path)
open(path, File::CREAT | File::EXCL | File::WRONLY) do |pid|
pid.flock(File::LOCK_EX)
pid.puts Process.pid
pid.flock(File::LOCK_UN)
end
at_exit do
remove(path)
end
true
rescue Errno::EEXIST # file already exists
false
end
def remove(path)
File.unlink(path)
true
rescue Exception
false
end
end

9. PID FILES IN ACTION
require "pid_file"
if PIDFile.create("show_pid.pid")
puts "Running as #{Process.pid}..."
sleep 10
puts "Shutting down."
else
puts "Already running."
end

10. PID FILES IN ACTION
require "pid_file"
if PIDFile.create("show_pid.pid")
puts "Running as #{Process.pid}..."
sleep 10
puts "Shutting down."
else
puts "Already running."
end
$ ruby -I . show_pid.rb
Running as 3756...
Shutting down.

11. PID FILES IN ACTION
require "pid_file"
if PIDFile.create("show_pid.pid")
puts "Running as #{Process.pid}..."
sleep 10
puts "Shutting down."
else
puts "Already running."
end
$ ruby -I . show_pid.rb
Running as 3756...
Shutting down.
$ cat show_pid.pid
cat: show_pid.pid: No such file or directory
$ ruby -I . show_pid.rb
Already running.
$ cat show_pid.pid
3756
$ cat show_pid.pid
cat: show_pid.pid: No such file or directory

12. LONG RUNNING NATURE

13. LONG RUNNING PROCESSES
require "pid_file"
module LongRunningProcess
module_function
def manage(name, args = ARGV, &job)
pid_path = "/Users/james/Desktop/#{name}.pid"
if args.include? "stop"
if pid = File.read(pid_path).to_i rescue nil
Process.kill("TERM", pid)
else
puts "Not running."
end
else
if PIDFile.create(pid_path)
job.call
else
puts "Already running."
end
end
end
end

14. LONG RUNNING PROCESSES
IN ACTION
require "long_running_process"
LongRunningProcess.manage("show_long_running_process") do
puts "Running as #{Process.pid}..."
at_exit do
puts "Shutting down."
end
loop do
sleep
end
end

15. LONG RUNNING PROCESSES
IN ACTION
require "long_running_process"
LongRunningProcess.manage("show_long_running_process") do
puts "Running as #{Process.pid}..."
at_exit do
puts "Shutting down."
end
loop do
sleep
end $ ruby -I . show_long_running_process.rb start
end Running as 6970...
Shutting down.
Terminated

16. LONG RUNNING PROCESSES
IN ACTION
require "long_running_process"
LongRunningProcess.manage("show_long_running_process") do
puts "Running as #{Process.pid}..."
at_exit do
puts "Shutting down."
end
loop do
sleep
end $ ruby -I . show_long_running_process.rb start
end Running as 6970...
Shutting down.
Terminated
$ cat show_long_running_process.pid
cat: show_long_running_process.pid: No such file or directory
$ cat show_long_running_process.pid
6970
$ ruby -I . show_long_running_process.rb start
Already running.
$ ruby -I . show_long_running_process.rb stop
$ cat show_long_running_process.pid
cat: show_long_running_process.pid: No such file or directory

17. DAEMONS

18. SUPPORTING
DAEMONIZATION
def create(path, &daemonize)
open(path, File::CREAT | File::EXCL | File::WRONLY) do |pid|
pid.flock(File::LOCK_EX)
begin
if daemonize.nil? or (daemonize.call rescue false)
pid.puts Process.pid
else
return false
end
ensure
pid.flock(File::LOCK_UN)
end
end
at_exit do
remove(path)
end
true
rescue Errno::EEXIST # file already exists
false
end

19. DAEMONIZING
if PIDFile.create(pid_path) { Process.daemon }

20. A MINOR GOD

21. THE SMALLEST POSSIBLE GOD
DESKTOP = "/Users/james/Desktop"
RUBY = "ruby -I #{DESKTOP}"
God.watch do |w|
w.name = "show_long_running_process"
w.interval = 30.seconds
w.start = "#{RUBY} #{DESKTOP}/show_long_running_process.rb start"
w.stop = "#{RUBY} #{DESKTOP}/show_long_running_process.rb stop"
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.pid_file = "#{DESKTOP}/show_long_running_process.pid"
w.start_if do |start|
start.condition(:process_running) do |c|
c.interval = 5.seconds
c.running = false
end
end
end

22. IN ACTION
$ rvm 1.9.2
$ rvmsudo god -c minor_god.god -D
I [2011-01-13 15:42:43] INFO: Loading minor_god.god
I [2011-01-13 15:42:43] INFO: Syslog enabled.
I [2011-01-13 15:42:43] INFO: Using pid file directory: /var/run/god
I [2011-01-13 15:42:43] INFO: Started on drbunix:///tmp/god.17165.sock
I [2011-01-13 15:42:43] INFO: show_long_running_process move 'unmonitored' to 'up'
I [2011-01-13 15:42:43] INFO: show_long_running_process moved 'unmonitored' to 'up'
I [2011-01-13 15:42:43] INFO: show_long_running_process [trigger] process is not running
(ProcessRunning)
I [2011-01-13 15:42:43] INFO: show_long_running_process move 'up' to 'start'
I [2011-01-13 15:42:43] INFO: show_long_running_process start: ruby -I /Users/james/Desktop /
Users/james/Desktop/show_long_running_process.rb start
I [2011-01-13 15:42:53] INFO: show_long_running_process moved 'up' to 'up'
I [2011-01-13 15:42:53] INFO: show_long_running_process [ok] process is running
(ProcessRunning)
I [2011-01-13 15:42:58] INFO: show_long_running_process [ok] process is running
(ProcessRunning)
I [2011-01-13 15:43:03] INFO: show_long_running_process [ok] process is running
(ProcessRunning)
I [2011-01-13 15:43:09] INFO: show_long_running_process [ok] process is running
(ProcessRunning)
^C/Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:656:in `join': Interrupt
from /Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:656:in `start'
from /Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:667:in `at_exit'
from /Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:700:in `block in <top
(required)>'

23. DOING EVIL
UNDER GOD’S GAZE
$ rvmsudo god -c minor_god.god
$ ps auxww | grep show_long_running_process
root 1986 0.2 0.0 2448168 952 ?? S 3:56PM 0:00.02 ruby -
I /Users/james/Desktop /Users/james/Desktop/show_long_running_process.rb start
james 1988 0.0 0.0 2435116 532 s000 S+ 3:57PM 0:00.00 grep
show_long_running_process
$ sudo kill 1986
$ ps auxww | grep show_long_running_process
root 1996 0.2 0.0 2448168 936 ?? S 3:57PM 0:00.00 ruby -
I /Users/james/Desktop /Users/james/Desktop/show_long_running_process.rb start
james 1998 0.0 0.0 2435116 532 s000 S+ 3:57PM 0:00.00 grep
show_long_running_process
$ rvmsudo god terminate
..
Stopped all watches
Stopped god
$ ps auxww | grep show_long_running_process
james 2005 0.0 0.0 2435116 532 s000 S+ 3:57PM 0:00.00 grep
show_long_running_process

24. COMPLICATIONS OF GOD

25. CLEANING PID FILES
w.behavior(:clean_pid_file)

26. CONDITIONAL RESTARTS
w.restart_if do |restart|
restart.condition(:memory_usage) do |c|
c.above = 150.megabytes
c.times = [3, 5] # 3 out of 5 intervals
end
restart.condition(:cpu_usage) do |c|
c.above = 50.percent
c.times = 5
end
end

27. CONTROLLING DEATH
w.lifecycle do |on|
on.condition(:flapping) do |c|
c.to_state = [:start, :restart]
c.times = 5
c.within = 5.minute
c.transition = :unmonitored
c.retry_in = 10.minutes
c.retry_times = 5
c.retry_within = 2.hours
end
end

28. THE POWER OF GOD
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c| # start if process is not running
c.running = true w.transition(:up, :start) do |on|
end on.condition(:process_exits)
end
# failsafe
on.condition(:tries) do |c| # restart if memory or cpu is too high
c.times = 5 w.transition(:up, :restart) do |on|
c.transition = :start on.condition(:memory_usage) do |c|
end c.interval = 20
end c.above = 50.megabytes
c.times = [3, 5]
end
on.condition(:cpu_usage) do |c|
c.interval = 10
c.above = 10.percent
c.times = [3, 5]
end
end

29. WEIGHING AND
MEASURING GOD

30. HOW BIG IS GOD?
$ ps -p 2177 -o pid,%mem,rss,command
PID %MEM RSS COMMAND
2177 0.2 16624 /Users/james/.rvm/rubies/ruby-1.9.2-p136/bin/ruby /Users/jame
$ ps -p 2184 -o pid,%mem,rss,command
PID %MEM RSS COMMAND
2184 0.0 928 ruby -I /Users/james/Desktop /Users/james/Desktop/show_long_r

31. GOD’S JOY

32. GOD’S JOY
• Super easy gem install and setup

33. GOD’S JOY
• Super easy gem install and setup
• It works on Ruby 1.9.2

34. GOD’S JOY
• Super easy gem install and setup
• It works on Ruby 1.9.2
• Ruby DSL can be handy

35. GOD’S JOY
• Super easy gem install and setup
• It works on Ruby 1.9.2
• Ruby DSL can be handy
• Very configurable with lots of options

36. THE PRICE OF GOD

37. THE PRICE OF GOD
• Ruby DSL isn’t at all natural

38. THE PRICE OF GOD
• Ruby DSL isn’t at all natural
• God process in moderately expensive

39. THE PRICE OF GOD
• Ruby DSL isn’t at all natural
• God process in moderately expensive
• God leaked memory in the past (I think this is resolved now)