Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Dark Side of Monitoring

32 views

Published on

We all use monitoring - after all, we all want to ensure that our applications are working as expected. But can hackers use it to exploit our application? Join me to explore different ways to exploit application monitoring tools, and what mitigations we can use.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

The Dark Side of Monitoring

  1. 1. @omerlh The Dark Side of Monitoring Omer Levi Hevroni May 2019
  2. 2. @omerlh Observability https://www.navantis.com/managed-monitoring/
  3. 3. @omerlh Can Hackers Use It Too?
  4. 4. @omerlh I’m a builder @omerlh
  5. 5. @omerlh DevSecOps @
  6. 6. @omerlh Threat Modeling?
  7. 7. @omerlh Threat Modeling? ● What are we building? ● What can go wrong? ● What are we doing about it? ● Did we do a good job?
  8. 8. @omerlh What is Monitoring?
  9. 9. @omerlh
  10. 10. @omerlh
  11. 11. @omerlh Black Box Monitoring White Box Monitoring https://landing.google.com/sre/sre-book/chapters/monitoring-distributed-systems/
  12. 12. @omerlh Let’s Dive In!
  13. 13. @omerlh White Box Monitoring
  14. 14. @omerlh Threat Modeling ● What are we building? ● What can go wrong? ● What are we doing about it? ● Did we do a good job?
  15. 15. @omerlh White-box Monitoring Monitoring based on metrics exposed by the internals of the system, including logs, interfaces like the Java Virtual Machine Profiling Interface, or an HTTP handler that emits internal statistics.
  16. 16. @omerlh White-box Monitoring for Kubernetes ● Probes (liveness/readiness) ● Prometheus
  17. 17. @omerlh Probes
  18. 18. @omerlh What are we building?
  19. 19. @omerlh Probes
  20. 20. @omerlh Why? ●Check dependencies ○ Databases ○ Other micro-services ●Kill services if it misfunction before users experience it
  21. 21. @omerlh Code Example https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks
  22. 22. @omerlh What Can go wrong?
  23. 23. @omerlh Exploiting liveness - information disclosure
  24. 24. @omerlh Exploiting liveness – Denial of Service
  25. 25. @omerlh
  26. 26. @omerlh What are we doing about it?
  27. 27. @omerlh Block access https://www.edureka. co/community/19277/ access-some- specific-paths-while- using-kubernetes- ingress?show=19278 #a19278
  28. 28. @omerlh Caching Source code - Kamus
  29. 29. @omerlh Prometheus
  30. 30. @omerlh What are we building?
  31. 31. @omerlh Prometheus ● Monitoring system ● Time-Series database ● Alerting system ● Auto-discovery https://prometheus.io/
  32. 32. @omerlh Prometheus Scraping Model
  33. 33. @omerlh What Can go wrong?
  34. 34. @omerlh Exploiting Prometheus - information disclosure
  35. 35. @omerlh
  36. 36. @omerlh Some interesting metrics...
  37. 37. @omerlh What about this metric?
  38. 38. @omerlh Here is the code behind it…
  39. 39. @omerlh What Can We Do About It?
  40. 40. @omerlh Block access https://www.edureka. co/community/19277/ access-some- specific-paths-while- using-kubernetes- ingress?show=19278 #a19278
  41. 41. @omerlh Prometheus Metrics Limit https://www.omerlh.info/2019/03/04/keeping-prometheus-in-shape/
  42. 42. @omerlh Did we do a good job?
  43. 43. @omerlh Threat Modeling ● What are we building? ● What can go wrong? ● What are we doing about it? ● Did we do a good job?
  44. 44. @omerlh Black Box Monitoring White Box Monitoring https://landing.google.com/sre/sre- book/chapters/monitoring-distributed-systems/
  45. 45. @omerlh Movie Streaming
  46. 46. @omerlh What are we building?
  47. 47. @omerlh Black-box Monitoring Testing externally visible behavior as a user would see it.
  48. 48. @omerlh Our Monitoring System
  49. 49. @omerlh What Can go wrong?
  50. 50. @omerlh STRIDE S – Spoofing T – Tampering R – Repudiation I – Information Disclosure D – Denial of Service E – Elevation of Privileges
  51. 51. @omerlh Spoofing
  52. 52. @omerlh Information Disclosure
  53. 53. @omerlh Repudiation
  54. 54. @omerlh Denial of Service
  55. 55. @omerlh What are we doing about it?
  56. 56. @omerlh Potential Mitigations ● Least Privilege ● Block access ● Tracing ● Limit to test data only
  57. 57. @omerlh Did we do a good job?
  58. 58. @omerlh http://www.applestory.biz/hermione-hand-raise-gif.html Questions?
  59. 59. @omerlh Can Hackers Use It Too?
  60. 60. @omerlh Key Take Away ● Monitoring is just code ● Careful when exposed to the internet ● Use cache when possible ● Conduct threat model for black-box monitoring
  61. 61. @omerlh Feedback appreciated
  62. 62. @omerlh Observability https://www.navantis.com/managed-monitoring/
  63. 63. @omerlh Thank You! Omer Levi Hevroni April 2019

×