• Save
Monitoring with Syslog and EventMachine
Upcoming SlideShare
Loading in...5

Monitoring with Syslog and EventMachine



Go beyond server and application stack monitoring, collect application specific metrics and push the information to interested parties. With the help of syslog, EventMachine and a few lines of Ruby ...

Go beyond server and application stack monitoring, collect application specific metrics and push the information to interested parties. With the help of syslog, EventMachine and a few lines of Ruby code, you can build a language agnostic system to gather and process custom tailored reporting data.



Total Views
Views on SlideShare
Embed Views



5 Embeds 33

http://localhost 18
http://www.huesler-informatik.ch 12
http://a0.twimg.com 1
https://si0.twimg.com 1
https://twitter.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Thank you. It was very interesting, specially last slide was cool, If I had a chance to join your team i'd obviously join wooga.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Monitoring with Syslog and EventMachine Monitoring with Syslog and EventMachine Presentation Transcript

  • Monitoring with Syslog andEventMachine
  • About mePatrick HueslerTwitter/Github: phueslerdeveloper@woogaBerlin, Germany
  • Wooga Social Games3# game developer on facebook ~ 6.5 Million users per day
  • Monitoring with Syslog andEventMachine
  • monitor |ˈmänəәtəәr|verb [ with obj. ]observe and check the progress orquality of (something) over a period oftime; keep under systematic review:equipment was installed to monitor air quality. Oxford Dictionary
  • DashboardDashboardDashboard
  • Visualization of Ticket Sale
  • Concurrent users of Magic Land
  • Monster World Dashboard
  • Motivation
  • DevOps
  • Moving Target
  • Debugging
  • “Since the last deploy, the number of signup errorshas gone up by 300 %. We might have broken something.”
  • Money Hose
  • Counting Things
  • Concurrent Users
  • Logins
  • Signups
  • Errors
  • A rough sketch
  • Application Application Application Server Server Server Event Aggregation Dashboard
  • Criteria
  • Simple
  • Fire And Forget
  • Non blocking
  • CriteriaSimple/Lightweight Polyglot Fire and Forget
  • A Little Story
  • Not Supportednode.js NewRelic RPMerlang NewRelic RPM
  • Load Balancer haproxynode.js node.js node.js
  • haproxy logshaproxy[674]: [15/Oct/2003:08:32:17.654] px-http   px-http/srv1 9/0/7/14/+30 200 +243 - - ---- 3/3/3/1/0 0/0   "GET /image.iso HTTP/1.0"
  • send logs custom sysloghaproxy server NewRelic Ruby New Relic Agent
  • haproxy2rpmhttp://github.com/wooga/haproxy2rpm
  • Syslog
  • Syslog• Standard logging solution for Unix/Linux• Facility (daemon, cron, user, local0, etc.)• Priority/Level (Alert, Critical, Error, Warning, etc.)• Client and server• Since 1980
  • ... It also provides devices which would otherwise be unable tocommunicate a means to notify administrators of problems or performance. http://en.wikipedia.org/wiki/Syslog
  • Syslog FormatDate Hostname Program : MessageJan 1 12:12:12 foo[421] : this is a message
  • UDP
  • Fire And Forget
  • Checklist
  • Simple/Lightweight ✔ Polyglot ✔ Fire and Forget ✔
  • Build it
  • send messageEvent Source UDP : 514 Event Aggregator Dashboard
  • send messageEvent Source UDP : 514 Syslog forwardDashboard Event Aggregator
  • Event Source # man loggerlogger -p local0.notice -t HOSTIDM My Message
  • Event Sourcerequire syslogdef log(message, level = :warning)  script_name = $0  syslog_option = Syslog::LOG_PID | Syslog::LOG_CONS  Syslog.open($0, syslog_option) do |s|    s.send(level, message)  endend
  • Server• Listen on UDP• Or tail a log file• receive and parse syslog message• update a counter
  • Serverrequire eventmachineclass Handler < EM::Connection  def receive_data(data)  log_line = SyslogParser.parse(data)    puts log_line.message  endendEM.run {  host = ""  port = "3000"  EM::open_datagram_socket(host, port, Handler)}
  • Syslog Parserhttps://github.com/jordansissel/experiments/blob/master/ ruby/eventmachine-speed/basic.rb
  • Websocket
  • require em-websocketEM.run do  channel = EM::Channel.new  options = {:host => "", :port => 8080}  EM::WebSocket.start(options) do |ws|    ws.onopen do      sid = channel.subscribe{|msg| ws.send msg}    end    ws.onclose do      channel.unsubscribe(sid)    end  endend
  • Pass on EM:ChannelEM.run {  channel = EM::Channel.new  # define your websocket server here  # start udp server  host = ""  port = "3000"  # pass in the channel to the data handler  EM::open_datagram_socket(host, port, Handler,channel)}
  • New Handlerclass Handler < EM::Connection  def initialize(*args)   @channel = args[0]    @counter = 0    super *args  end  def receive_data(data)   if data && data.size > 0      @counter += 1     @channel.push(@counter)    end  endend
  • JavaScriptvar socket;var host = "ws://localhost:8080";var socket = new WebSocket(host);socket.onopen = function(){    console.log(open);}socket.onmessage = function(msg){    console.log(msg.data);}socket.onclose = function(){    console.log(closed);}
  • Server-sent events• Push only• http://dev.w3.org/html5/eventsource/• http://en.wikipedia.org/wiki/Server- sent_events• http://www.html5rocks.com/en/tutorials/ casestudies/sunlight_streamcongress.html
  • What else?
  • Graphinghttp://graphite.wikidot.com/screen-shots
  • Steal from statsd https://github.com/etsy/statsd
  • Inspiration• http://codeascraft.etsy.com/2011/02/15/ measure-anything-measure-everything/• http://code.flickr.com/blog/2008/10/27/ counting-timing/
  • Thank You
  • Special thanks to• @wooga for supporting me• @knutin for facebook faces dashboard• @hukl for hummingbird video (http:// vimeo.com/20840998)
  • Slideshttp://www.slideshare.net/wooga
  • wooga.com/jobs WANTED dead or alive BACKEND DEVELOPER REWARD www.wooga.com/ jobs/