Counting on God

COUNTING ON GOD Can we?

GOD’S CHILDREN

ELEMENTS OFMONITORED PROCESSES

ELEMENTS OF MONITORED PROCESSES• PID ﬁles

ELEMENTS OF MONITORED PROCESSES• PID ﬁles• Long running nature (decoupled start/stop)

ELEMENTS OF MONITORED PROCESSES• PID ﬁles• Long running nature (decoupled start/stop)• Daemonization

PID FILES

PID FILE BASICSmodule PIDFile module_function def create(path) open(path, File::CREAT | File::EXCL | File::WRONLY) do |pid| pid.flock(File::LOCK_EX) pid.puts Process.pid pid.flock(File::LOCK_UN) end at_exit do remove(path) end true rescue Errno::EEXIST # file already exists false end def remove(path) File.unlink(path) true rescue Exception false endend

PID FILES IN ACTIONrequire "pid_file"if PIDFile.create("show_pid.pid") puts "Running as #{Process.pid}..." sleep 10 puts "Shutting down."else puts "Already running."end

PID FILES IN ACTIONrequire "pid_file"if PIDFile.create("show_pid.pid") puts "Running as #{Process.pid}..." sleep 10 puts "Shutting down."else puts "Already running."end $ ruby -I . show_pid.rb Running as 3756... Shutting down.

PID FILES IN ACTIONrequire "pid_file"if PIDFile.create("show_pid.pid") puts "Running as #{Process.pid}..." sleep 10 puts "Shutting down."else puts "Already running."end $ ruby -I . show_pid.rb Running as 3756... Shutting down. $ cat show_pid.pid cat: show_pid.pid: No such file or directory $ ruby -I . show_pid.rb Already running. $ cat show_pid.pid 3756 $ cat show_pid.pid cat: show_pid.pid: No such file or directory

LONG RUNNING NATURE

LONG RUNNING PROCESSES require "pid_file" module LongRunningProcess module_function def manage(name, args = ARGV, &job) pid_path = "/Users/james/Desktop/#{name}.pid" if args.include? "stop" if pid = File.read(pid_path).to_i rescue nil Process.kill("TERM", pid) else puts "Not running." end else if PIDFile.create(pid_path) job.call else puts "Already running." end end end end

LONG RUNNING PROCESSES IN ACTIONrequire "long_running_process"LongRunningProcess.manage("show_long_running_process") do puts "Running as #{Process.pid}..." at_exit do puts "Shutting down." end loop do sleep endend

LONG RUNNING PROCESSES IN ACTIONrequire "long_running_process"LongRunningProcess.manage("show_long_running_process") do puts "Running as #{Process.pid}..." at_exit do puts "Shutting down." end loop do sleep end $ ruby -I . show_long_running_process.rb startend Running as 6970... Shutting down. Terminated

LONG RUNNING PROCESSES IN ACTIONrequire "long_running_process"LongRunningProcess.manage("show_long_running_process") do puts "Running as #{Process.pid}..." at_exit do puts "Shutting down." end loop do sleep end $ ruby -I . show_long_running_process.rb startend Running as 6970... Shutting down. Terminated $ cat show_long_running_process.pid cat: show_long_running_process.pid: No such file or directory $ cat show_long_running_process.pid 6970 $ ruby -I . show_long_running_process.rb start Already running. $ ruby -I . show_long_running_process.rb stop $ cat show_long_running_process.pid cat: show_long_running_process.pid: No such file or directory

DAEMONS

SUPPORTING DAEMONIZATIONdef create(path, &daemonize) open(path, File::CREAT | File::EXCL | File::WRONLY) do |pid| pid.flock(File::LOCK_EX) begin if daemonize.nil? or (daemonize.call rescue false) pid.puts Process.pid else return false end ensure pid.flock(File::LOCK_UN) end end at_exit do remove(path) end truerescue Errno::EEXIST # file already exists falseend

DAEMONIZINGif PIDFile.create(pid_path) { Process.daemon }

A MINOR GOD

THE SMALLEST POSSIBLE GOD DESKTOP = "/Users/james/Desktop" RUBY = "ruby -I #{DESKTOP}" God.watch do |w| w.name = "show_long_running_process" w.interval = 30.seconds w.start = "#{RUBY} #{DESKTOP}/show_long_running_process.rb start" w.stop = "#{RUBY} #{DESKTOP}/show_long_running_process.rb stop" w.start_grace = 10.seconds w.restart_grace = 10.seconds w.pid_file = "#{DESKTOP}/show_long_running_process.pid" w.start_if do |start| start.condition(:process_running) do |c| c.interval = 5.seconds c.running = false end end end

IN ACTION$ rvm 1.9.2$ rvmsudo god -c minor_god.god -DI [2011-01-13 15:42:43] INFO: Loading minor_god.godI [2011-01-13 15:42:43] INFO: Syslog enabled.I [2011-01-13 15:42:43] INFO: Using pid file directory: /var/run/godI [2011-01-13 15:42:43] INFO: Started on drbunix:///tmp/god.17165.sockI [2011-01-13 15:42:43] INFO: show_long_running_process move unmonitored to upI [2011-01-13 15:42:43] INFO: show_long_running_process moved unmonitored to upI [2011-01-13 15:42:43] INFO: show_long_running_process [trigger] process is not running(ProcessRunning)I [2011-01-13 15:42:43] INFO: show_long_running_process move up to startI [2011-01-13 15:42:43] INFO: show_long_running_process start: ruby -I /Users/james/Desktop /Users/james/Desktop/show_long_running_process.rb startI [2011-01-13 15:42:53] INFO: show_long_running_process moved up to upI [2011-01-13 15:42:53] INFO: show_long_running_process [ok] process is running(ProcessRunning)I [2011-01-13 15:42:58] INFO: show_long_running_process [ok] process is running(ProcessRunning)I [2011-01-13 15:43:03] INFO: show_long_running_process [ok] process is running(ProcessRunning)I [2011-01-13 15:43:09] INFO: show_long_running_process [ok] process is running(ProcessRunning)^C/Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:656:in `join: Interrupt from /Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:656:in `start from /Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:667:in `at_exit from /Users/james/.rvm/gems/ruby-1.9.2-p136/gems/god-0.11.0/lib/god.rb:700:in `block in <top(required)>

DOING EVIL UNDER GOD’S GAZE$ rvmsudo god -c minor_god.god$ ps auxww | grep show_long_running_processroot 1986 0.2 0.0 2448168 952 ?? S 3:56PM 0:00.02 ruby -I /Users/james/Desktop /Users/james/Desktop/show_long_running_process.rb startjames 1988 0.0 0.0 2435116 532 s000 S+ 3:57PM 0:00.00 grepshow_long_running_process$ sudo kill 1986$ ps auxww | grep show_long_running_processroot 1996 0.2 0.0 2448168 936 ?? S 3:57PM 0:00.00 ruby -I /Users/james/Desktop /Users/james/Desktop/show_long_running_process.rb startjames 1998 0.0 0.0 2435116 532 s000 S+ 3:57PM 0:00.00 grepshow_long_running_process$ rvmsudo god terminate..Stopped all watchesStopped god$ ps auxww | grep show_long_running_processjames 2005 0.0 0.0 2435116 532 s000 S+ 3:57PM 0:00.00 grepshow_long_running_process

COMPLICATIONS OF GOD

CLEANING PID FILES w.behavior(:clean_pid_file)

CONTROLLING DEATH w.lifecycle do |on| on.condition(:flapping) do |c| c.to_state = [:start, :restart] c.times = 5 c.within = 5.minute c.transition = :unmonitored c.retry_in = 10.minutes c.retry_times = 5 c.retry_within = 2.hours end end

THE POWER OF GOD# determine the state on startupw.transition(:init, { true => :up, false => :start }) do |on| on.condition(:process_running) do |c| c.running = true endend# determine when process has finished startingw.transition([:start, :restart], :up) do |on| on.condition(:process_running) do |c| # start if process is not running c.running = true w.transition(:up, :start) do |on| end on.condition(:process_exits) end # failsafe on.condition(:tries) do |c| # restart if memory or cpu is too high c.times = 5 w.transition(:up, :restart) do |on| c.transition = :start on.condition(:memory_usage) do |c| end c.interval = 20end c.above = 50.megabytes c.times = [3, 5] end on.condition(:cpu_usage) do |c| c.interval = 10 c.above = 10.percent c.times = [3, 5] end end

WEIGHING ANDMEASURING GOD

HOW BIG IS GOD?$ ps -p 2177 -o pid,%mem,rss,command PID %MEM RSS COMMAND 2177 0.2 16624 /Users/james/.rvm/rubies/ruby-1.9.2-p136/bin/ruby /Users/jame$ ps -p 2184 -o pid,%mem,rss,command PID %MEM RSS COMMAND 2184 0.0 928 ruby -I /Users/james/Desktop /Users/james/Desktop/show_long_r

GOD’S JOY

GOD’S JOY• Super easy gem install and setup

GOD’S JOY• Super easy gem install and setup• It works on Ruby 1.9.2

GOD’S JOY• Super easy gem install and setup• It works on Ruby 1.9.2• Ruby DSL can be handy

GOD’S JOY• Super easy gem install and setup• It works on Ruby 1.9.2• Ruby DSL can be handy• Very conﬁgurable with lots of options

THE PRICE OF GOD

THE PRICE OF GOD• Ruby DSL isn’t at all natural

THE PRICE OF GOD• Ruby DSL isn’t at all natural• God process in moderately expensive

THE PRICE OF GOD• Ruby DSL isn’t at all natural• God process in moderately expensive• God leaked memory in the past (I think this is resolved now)

Counting on God

by James Gray on Jan 13, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Counting on God — Presentation Transcript