Successfully reported this slideshow.
Real-Time-MonitoringMax Köhler I 9. Feb. 2012                            © 2012 Mayflower GmbH
Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 2
Real-Time-Monitoring
... what ?
monitoring sucks            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 5
„    But does monitoring       REALLY suck?                 Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 6
...      Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 7
„Yes, yes i know:       You can just combine Nagios + collectd + graphite +       pnp4nagios an you have everything you ne...
„Seriously? Kiss my ass. I‘m a huge fan of the Unix pipelinephilosophy but, crhist, have you ever heard the phrase„antipat...
https://github.com/monitoringsucks                      Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 10
monitoring sucks            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 11
monitoring sucks            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 11
monitoring sucks                 is fun            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 11
its all about ...              Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 12
Making Decisions           Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 13
Making Decisions           Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 14
How many users are using     feature xyz ?               Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 15
Feature #xyz34%          Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 16
Did we make it faster?               Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 17
base          feature xyz400ms300ms200ms100ms 0ms        5.2     6.2   7.2             8.2                         9.2    ...
No !       Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 19
Can we deploy that   for all users?            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 20
Did we break something?         Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 21
events / sec60453015 0  14:00   15:00       16:00          17:00                        18:00                             ...
Yes !        Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 23
Will we even notice?              Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 24
Real-Time-Monitoring I   Mayflower GmbH I 9. Feb 2012 I 25
Changes          http://www.flickr.com/photos/addrox/2587484034/sizes/m/in/photostream/                             Real-Ti...
Monitor!Monitor!Monitor!
„If you can not measure it, you can not improve it!“                                                    Lord Kelvin       ...
DevOp                                                       bingo ;)        Real-Time-Monitoring I   Mayflower GmbH I 9. Fe...
Access                Yes                                             Nohttp://www.flickr.com/photos/mogwai_83/3022261893/ ...
Tooling          Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 31
Carbon                        StatsD               WhisperGraphite                                 Logster                ...
GraphiteEnterprise Scalable Realtime Graphing                         Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 201...
1. Store numeric time-series data2. Render graphs on demand                      Real-Time-Monitoring I   Mayflower GmbH I ...
Graphite                                          Data               rendering                  carbon     browser        ...
StatsD         Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 36
Graphite                                               statsd               rendering                  carbon     browser ...
CountingStatsD::increment(events.success); StatsD::decrement(event.count);                         Real-Time-Monitoring I ...
Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 39
Timing$timeStart = microtime(true);$worker->process();$timeEnd = round(microtime(true) - $timeStart, 4);StatsD::timing(eve...
Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 41
SamplingStatsD::increment(events.success, 0.1);                              Real-Time-Monitoring I   Mayflower GmbH I 9. F...
gdash        Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 43
Dashboard// dash.yaml:name: Event-Queue:description: Informationen über den Gesundheitszustanddes Patienten...            ...
Graphs// timing.graphtitle       "Event-Queue Timing "vtitle      "Duration in s"from        "-1hours"area        :nonelin...
Graph Settingstitle         "A sample graph"vtitle        "events"width         800height        600from          "-2days"...
Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 47
Logster          Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 48
Log2012/02/02 09:09:06 [warning] 13674#0: *24533 FastCGI sent in stderr: ...2012/02/02 14:06:07 [warning] 13674#0: *24533 ...
Log2012/02/02 09:09:06 [warning] 13674#0: *24533 FastCGI sent in stderr: ...2012/02/02 14:06:07 [warning] 13674#0: *24533 ...
Parser         ....self.reg = re.compile(magic)         ....# Return a list of metrics objectsreturn [    MetricObject("wa...
600450300150  0      12   13   14   15   16   17   18   19   20   21      22       23      24      25     26              ...
Deploy-Trendecho "deploys 1 `date +%s`" | nc graphite.info 2003                                     Real-Time-Monitoring I...
600450300150  0      12   13   14   15   16   17   18   19   20   21      22       23      24      25     26              ...
Happy Developers600450300150  0      12   13   14   15   16   17   18   19   20   21      22       23      24      25     ...
CollectD-Graphite      Plugin            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 55
collectd to graphite#collectd.conf<Plugin write_http>  <URL "http://monitor:3012/post-collectd">  </URL></Plugin>         ...
Start !          Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 57
Graphitehttp://gist.github.com/1023728                        Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 58
StatsDgit clone https://github.com/etsy/statsd.git                                Real-Time-Monitoring I   Mayflower GmbH I...
StatsD Client            Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 60
Vagrant          Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. 2012 I 61
Making Decisions  Low MTTD   (Mean Time To Detect)                       Real-Time-Monitoring I   Mayflower GmbH I 9. Feb. ...
Vielen Dank für Ihre Aufmerksamkeit!Kontakt   Max Köhler          max.koehler@mayflower.de          +49 89 242054-1160     ...
Upcoming SlideShare
Loading in …5
×

Max Köhler - Real-Time-Monitoring

2,813 views

Published on

Jeder kennt Aussagen wie: Haben wir es schneller gemacht? Bringt uns das
neue Feature mehr Geld oder Besucher? Haben wir es kaputt gemacht und wann?
All diese Fragen können mittels Monitoring, implementiert durch nur
wenigen Zeilen Code, einfach beantwortet werden.
Anhand von Beispielen soll in dem Vortrag aufgezeigt werden, wie einfach
man mit Graphite, StatsD, gDash und weiteren Tools ein Monitoring
aufbauen kann das den kompletten Stack der Applikation abdeckt.

Published in: Technology, Business
  • Be the first to comment

Max Köhler - Real-Time-Monitoring

  1. 1. Real-Time-MonitoringMax Köhler I 9. Feb. 2012 © 2012 Mayflower GmbH
  2. 2. Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 2
  3. 3. Real-Time-Monitoring
  4. 4. ... what ?
  5. 5. monitoring sucks Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 5
  6. 6. „ But does monitoring REALLY suck? Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 6
  7. 7. ... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 7
  8. 8. „Yes, yes i know: You can just combine Nagios + collectd + graphite + pnp4nagios an you have everything you need! ... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 8
  9. 9. „Seriously? Kiss my ass. I‘m a huge fan of the Unix pipelinephilosophy but, crhist, have you ever heard the phrase„antipattern“? - John E. Vincenthttp://lusislog.blogspot.com/2011/06/why-monitoring-sucks.html Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 9
  10. 10. https://github.com/monitoringsucks Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 10
  11. 11. monitoring sucks Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 11
  12. 12. monitoring sucks Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 11
  13. 13. monitoring sucks is fun Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 11
  14. 14. its all about ... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 12
  15. 15. Making Decisions Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 13
  16. 16. Making Decisions Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 14
  17. 17. How many users are using feature xyz ? Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 15
  18. 18. Feature #xyz34% Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 16
  19. 19. Did we make it faster? Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 17
  20. 20. base feature xyz400ms300ms200ms100ms 0ms 5.2 6.2 7.2 8.2 9.2 Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 18
  21. 21. No ! Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 19
  22. 22. Can we deploy that for all users? Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 20
  23. 23. Did we break something? Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 21
  24. 24. events / sec60453015 0 14:00 15:00 16:00 17:00 18:00 Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 22
  25. 25. Yes ! Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 23
  26. 26. Will we even notice? Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 24
  27. 27. Real-Time-Monitoring I Mayflower GmbH I 9. Feb 2012 I 25
  28. 28. Changes http://www.flickr.com/photos/addrox/2587484034/sizes/m/in/photostream/ Real-Time-Monitoring I Mayflower GmbH I 9. Feb 2012 I 26
  29. 29. Monitor!Monitor!Monitor!
  30. 30. „If you can not measure it, you can not improve it!“ Lord Kelvin Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 28
  31. 31. DevOp bingo ;) Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 29
  32. 32. Access Yes Nohttp://www.flickr.com/photos/mogwai_83/3022261893/ Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 30
  33. 33. Tooling Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 31
  34. 34. Carbon StatsD WhisperGraphite Logster Real-Time CollectD Low-Level High-Level Dashboards gdash Graphiti Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 32
  35. 35. GraphiteEnterprise Scalable Realtime Graphing Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 33
  36. 36. 1. Store numeric time-series data2. Render graphs on demand Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 34
  37. 37. Graphite Data rendering carbon browser whisper Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 35
  38. 38. StatsD Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 36
  39. 39. Graphite statsd rendering carbon browser whisper Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 37
  40. 40. CountingStatsD::increment(events.success); StatsD::decrement(event.count); Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 38
  41. 41. Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 39
  42. 42. Timing$timeStart = microtime(true);$worker->process();$timeEnd = round(microtime(true) - $timeStart, 4);StatsD::timing(events.timing, $timeEnd); Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 40
  43. 43. Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 41
  44. 44. SamplingStatsD::increment(events.success, 0.1); Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 42
  45. 45. gdash Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 43
  46. 46. Dashboard// dash.yaml:name: Event-Queue:description: Informationen über den Gesundheitszustanddes Patienten... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 44
  47. 47. Graphs// timing.graphtitle "Event-Queue Timing "vtitle "Duration in s"from "-1hours"area :nonelinewidth 1description "Duration in s"field :lower, :scale => 1, :color => "green", :alias => "Lower", :data => "sumSeries(events.*.timing.lower)", :derivative => falsefield :mean, :scale => 1, :color => "yellow", :alias => "Mean", :data => "sumSeries(events.*.timing.mean)", :derivative => false Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 45
  48. 48. Graph Settingstitle "A sample graph"vtitle "events"width 800height 600from "-2days"until "-1days"area :all|first|nonedescription "Events"hide_legend truelinewidth 2... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 46
  49. 49. Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 47
  50. 50. Logster Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 48
  51. 51. Log2012/02/02 09:09:06 [warning] 13674#0: *24533 FastCGI sent in stderr: ...2012/02/02 14:06:07 [warning] 13674#0: *24533 FastCGI sent in stderr: ...2012/02/02 19:02:01 [warning] 13674#0: *24533 FastCGI sent in stderr: ... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 49
  52. 52. Log2012/02/02 09:09:06 [warning] 13674#0: *24533 FastCGI sent in stderr: ...2012/02/02 14:06:07 [warning] 13674#0: *24533 FastCGI sent in stderr: ...2012/02/02 19:02:01 [warning] 13674#0: *24533 FastCGI sent in stderr: ... Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 50
  53. 53. Parser ....self.reg = re.compile(magic) ....# Return a list of metrics objectsreturn [    MetricObject("warning", (self.warning / self.duration), "warnings per sec"),] Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 51
  54. 54. 600450300150 0 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Real-Time-Monitoring I Mayflower GmbH I 9. Feb 2012 I 52
  55. 55. Deploy-Trendecho "deploys 1 `date +%s`" | nc graphite.info 2003 Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 53
  56. 56. 600450300150 0 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Real-Time-Monitoring I Mayflower GmbH I 9. Feb 2012 I 54
  57. 57. Happy Developers600450300150 0 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Real-Time-Monitoring I Mayflower GmbH I 9. Feb 2012 I 54
  58. 58. CollectD-Graphite Plugin Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 55
  59. 59. collectd to graphite#collectd.conf<Plugin write_http> <URL "http://monitor:3012/post-collectd"> </URL></Plugin> Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 56
  60. 60. Start ! Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 57
  61. 61. Graphitehttp://gist.github.com/1023728 Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 58
  62. 62. StatsDgit clone https://github.com/etsy/statsd.git Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 59
  63. 63. StatsD Client Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 60
  64. 64. Vagrant Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 61
  65. 65. Making Decisions Low MTTD (Mean Time To Detect) Real-Time-Monitoring I Mayflower GmbH I 9. Feb. 2012 I 62
  66. 66. Vielen Dank für Ihre Aufmerksamkeit!Kontakt Max Köhler max.koehler@mayflower.de +49 89 242054-1160 Mayflower GmbH Mannhardtstr. 6 80538 München © 2012 Mayflower GmbH

×