Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Can skynet beat humans in signal detection?

446 views

Published on

Finding patterns in high-resolution system monitoring data is hard. Computers are good at some jobs, humans others. Computers can pre-process the data so people can monitor databases more effectively.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Can skynet beat humans in signal detection?

  1. 1. Can Skynet Beat Humans in Signal Detection?
  2. 2. Background Database Performance Management Measure ‘All the Things’ in 1-Second Detail
  3. 3. Your Mandate Manage 300x more data with 1.5x more people.
  4. 4. Signal Detection Theory Humans process up to 500GB of data per second.
  5. 5. Find the Outliers
  6. 6. Signal Detection Easy Hard
  7. 7. Easy Signal Detection Easy signals are always easy, regardless of set size. 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 200 300 400 500 600 700 800 900 1000 % Correct Response Time Set Size Set Size
  8. 8. Hard Signal Detection Hard signals get harder as set size increases. 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 200 300 400 500 600 700 800 900 1000 % Correct Response Time Set SizeSet Size
  9. 9. Signal Detection in Use Statistical Process Controls are “Easy” Upper Control Limit Lowe Control Limit
  10. 10. Signal Detection in Systems 1-Second CPU over 5 Minutes What’s the cause?
  11. 11. Looking at Work In a database, a query is work. But, I have 300 query classes. Crap.
  12. 12. Graph Everything Query Execution Time CPU
  13. 13. Reduce Data Zoom & Remove the Obvious 1/10th of Data Points Query Execution Time CPU
  14. 14. Man vs Machine Man Machine
  15. 15. Man vs Machine Man Machine Non-linear increasing CPU Statistically significant
  16. 16. Man vs Machine Man Machine Non-linear increasing CPU Statistically significant Pattern! Not model friendlyPattern! Pattern!
  17. 17. Shortcomings Elephants in the room?
  18. 18. Shortcomings Humans lose with a known pattern
  19. 19. Key Thoughts Our brains do much better with fewer distractors, so eliminate them! With unknown patterns, humans find signals faster than computers.
  20. 20. Thanks http://www.flickr.com/photos/danielproulx/3524826318/ http://www.sciencedirect.com/science/article/pii/S0896627301003920 http://www.himandus.net/elefunteria/eday/puzzles/hunt_24.html http://www.federaljack.com/terminator-robots-a-reality/ Remember: Less is more.

×