Your SlideShare is downloading. ×
Latency: The Silent Monitoring System Killer
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Latency: The Silent Monitoring System Killer

1,774
views

Published on

Slides from a lighting talk presented at the January Sydney DevOps Meetup.

Slides from a lighting talk presented at the January Sydney DevOps Meetup.

Published in: Technology, Business

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,774
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. LATENCY THE SILENT MONITORING SYSTEM KILLERSaturday, 21 January 2012
  • 2. #MONITORINGSUCKSSaturday, 21 January 2012
  • 3. SCALABILITYSaturday, 21 January 2012
  • 4. execute large volumes of monitoring checks under a variety of conditions (good + bad) with a consistent throughputSaturday, 21 January 2012
  • 5. CONSISTENT THROUGHPUTSaturday, 21 January 2012
  • 6. SHORT CHECK EXECUTION TIMESaturday, 21 January 2012
  • 7. WHAT INTRODUCES VARIABILITY?Saturday, 21 January 2012
  • 8. LATENCY INTRODUCED c SYNCHRONOUS CALLSSaturday, 21 January 2012
  • 9. Saturday, 21 January 2012
  • 10. = 1 monitoring checkSaturday, 21 January 2012
  • 11. Saturday, 21 January 2012
  • 12. 150 monitoring checks each executed every 300 seconds each takes 1 second checks are executed seriallySaturday, 21 January 2012
  • 13. all checks executed in 150 seconds monitoring system at 50% capacitySaturday, 21 January 2012
  • 14. DOUBLE THE CHECKSSaturday, 21 January 2012
  • 15. Saturday, 21 January 2012
  • 16. all checks executed in 300 seconds monitoring system at 100% capacitySaturday, 21 January 2012
  • 17. DOUBLE THE EXECUTION TIMESaturday, 21 January 2012
  • 18. Saturday, 21 January 2012
  • 19. all checks executed in 600 seconds monitoring system at 200% capacity only 50% of the checks are “on time”Saturday, 21 January 2012
  • 20. CHECK LATENCYSaturday, 21 January 2012
  • 21. Saturday, 21 January 2012
  • 22. HOW DO WE FIX THIS!?Saturday, 21 January 2012
  • 23. PERFORMANCE ANALYSIS!Saturday, 21 January 2012
  • 24. AN ANALOGYSaturday, 21 January 2012
  • 25. MONITORING CHECK == “ACTION” ON MVC WEB APPSaturday, 21 January 2012
  • 26. Saturday, 21 January 2012
  • 27. Saturday, 21 January 2012
  • 28. SEPARATE DATA COLLECTION FROM THRESHOLDING f NOTIFICATIONSSaturday, 21 January 2012
  • 29. Saturday, 21 January 2012
  • 30. THIS SHIFTS LATENCYSaturday, 21 January 2012
  • 31. IT DOES NOT ELIMINATE IT!Saturday, 21 January 2012
  • 32. Saturday, 21 January 2012
  • 33. RRDTOOL IS EVILSaturday, 21 January 2012
  • 34. USE SOMETHING BETTER!Saturday, 21 January 2012
  • 35. USE OPENTSDB OR GANGLIA WITH CHECK_TSDB OR CHECK_GMONDSaturday, 21 January 2012
  • 36. Saturday, 21 January 2012
  • 37. A DIFFERENT SET a PROBLEMSSaturday, 21 January 2012
  • 38. STORAGE WILL GO AWAYSaturday, 21 January 2012
  • 39. CHAOS WILL ENSUESaturday, 21 January 2012
  • 40. PAGERS WILL MELTSaturday, 21 January 2012
  • 41. SET UP “META- PARENTING”Saturday, 21 January 2012
  • 42. BUILD A KILL SWITCHSaturday, 21 January 2012
  • 43. Saturday, 21 January 2012
  • 44. READ MORE ABOUT THIS:Saturday, 21 January 2012
  • 45. bit.ly/yN4mdySaturday, 21 January 2012

×