Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The DevOps Toolbox: Open Source Log Analytics

2,529 views

Published on

Logz.io cofounders Tomer Levy and Asaf Yigal discuss how to do log analytics with the ELK stack at a ‪DevOps‬ meetup held by Akamai Technologies.

Published in: Software
  • Be the first to comment

The DevOps Toolbox: Open Source Log Analytics

  1. 1. logz.io | @logzio | @tomerlevy | @asafyigal The DevOps Toolbox: Open Source Log Analytics Tomer Levy & Asaf Yigal Cofounders, Logz.io
  2. 2. logz.io | @logzio | @tomerlevy | @asafyigal Is anyone using ELK to process logs?
  3. 3. logz.io | @logzio | @tomerlevy | @asafyigal Is anyone using the public cloud? AWS?
  4. 4. logz.io | @logzio | @tomerlevy | @asafyigal Is anyone doing kite-surfing? Windsurfing? Paragliding? Sailing?
  5. 5. logz.io | @logzio | @tomerlevy | @asafyigal ELK – What is it good for?
  6. 6. logz.io | @logzio | @tomerlevy | @asafyigal This is our office
  7. 7. logz.io | @logzio | @tomerlevy | @asafyigal And this is the view from the office
  8. 8. logz.io | @logzio | @tomerlevy | @asafyigal When should we leave everything and go kitesurfing? This is our challenge
  9. 9. logz.io | @logzio | @tomerlevy | @asafyigal Wind analytics
  10. 10. logz.io | @logzio | @tomerlevy | @asafyigal Wind across locations
  11. 11. logz.io | @logzio | @tomerlevy | @asafyigal Next steps
  12. 12. logz.io | @logzio | @tomerlevy | @asafyigal Wind analytics — next items • Alerts • Wind forecast combined with real wind • Wind predictions!
  13. 13. logz.io | @logzio | @tomerlevy | @asafyigal Who are we? • Logz.io Insights: Behavioural intelligence to pinpoint what actually matters in logs • ELK++ as a Service – Infinitely scalable – Secured – Highly Available – Additional Features (alerts, role-based access)
  14. 14. logz.io | @logzio | @tomerlevy | @asafyigal ELK implementation — creating the right architecture
  15. 15. logz.io | @logzio | @tomerlevy | @asafyigal ELK implementation — creating the right architecture Curator Curator 3x Master Nodes + 1 data
  16. 16. logz.io | @logzio | @tomerlevy | @asafyigal ELK implementation — creating the right architecture Curator 3x Master Nodes + 1 data Index Failures Handler
  17. 17. logz.io | @logzio | @tomerlevy | @asafyigal ELK basic implementation — find the weak spots AZ-1 AZ-2 ELBLB
  18. 18. logz.io | @logzio | @tomerlevy | @asafyigal • Grok – parse logs to extract the relevant fields… • Try our blog for some help on grok/plugins etc’ • blog.logz.io ELK basic implementation — configuration
  19. 19. logz.io | @logzio | @tomerlevy | @asafyigal 1. Use Elasticsearch AWS Plugin 2. EBS are challenging for big environment  Use PIOPS if you can afford 3. Don’t run AWS cluster on the same AZ (but don’t run them on different zones!) - use Shard allocation awareness 4. S3 Snapshots are cool! Things tend to break… Elasticsearch basic implementation — configuration
  20. 20. logz.io | @logzio | @tomerlevy | @asafyigal Let’s see a high-level view of how we process logs Demo
  21. 21. logz.io | @logzio | @tomerlevy | @asafyigal Want to try our product? • Email us with questions: tomer@logz.io asaf@logz.io • Visit our website for more information!
  22. 22. logz.io | @logzio | @tomerlevy | @asafyigal We’re hiring — big time! • Elasticsearch experts • Java developers • Machine-learning experts • See our job listings page

×