God is watching over my Rails

On my home server I’m toying with some Rails clustering and such, using nginx and thin, and now God to watch over things.
With this setup, I can add apps and port-ranges to monitor any time I like, and I can split off a particularly specific configuration group if an app has unusual trends or requirements, such as using loads of memory.

RAILS_ROOT = "/var/www/rails"

APPS = {
  "redmine" => %w{5000 5001 5002 5003 5004},
  "typo5" => %w{4000 4001 4002}
  }

for app in APPS.keys
  for port in APPS[app]
    God.watch do |w|
      w.name = "#{app}-thin-#{port}"
      w.interval = 30.seconds # default
      w.start = "thin start -C #{RAILS_ROOT}/#{app}/config/thin.yml -o #{port}"
      w.stop = "thin stop -C #{RAILS_ROOT}/#{app}/config/thin.yml -o #{port}"
      w.restart = "thin restart -C #{RAILS_ROOT}/#{app}/config/thin.yml -o #{port}"
      w.start_grace = 10.seconds
      w.restart_grace = 10.seconds
      w.pid_file = File.join(RAILS_ROOT, "/#{app}/tmp/pids/thin.#{port}.pid")

      w.behavior(:clean_pid_file)

      w.start_if do |start|
        start.condition(:process_running) do |c|
          c.interval = 5.seconds
          c.running = false
        end
      end

      w.restart_if do |restart|
        restart.condition(:memory_usage) do |c|
          c.above = 80.megabytes
          c.times = [3, 5] # 3 out of 5 intervals
        end

        restart.condition(:cpu_usage) do |c|
          c.above = 50.percent
          c.times = 3
        end
      end

      # lifecycle
      w.lifecycle do |on|
        on.condition(:flapping) do |c|
          c.to_state = [:start, :restart]
          c.times = 5
          c.within = 5.minute
          c.transition = :unmonitored
          c.retry_in = 10.minutes
          c.retry_times = 5
          c.retry_within = 2.hours
        end
      end
    end
  end
end

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.