Logz.io cofounders Tomer Levy and Asaf Yigal discuss how to do log analytics with the ELK stack at a ‪DevOps‬ meetup held by Akamai Technologies.

1. logz.io | @logzio | @tomerlevy | @asafyigal
The DevOps Toolbox:
Open Source Log Analytics
Tomer Levy & Asaf Yigal
Cofounders, Logz.io

2. logz.io | @logzio | @tomerlevy | @asafyigal
Is anyone using ELK to process logs?

3. logz.io | @logzio | @tomerlevy | @asafyigal
Is anyone using the public cloud? AWS?

4. logz.io | @logzio | @tomerlevy | @asafyigal
Is anyone doing kite-surfing?
Windsurfing?
Paragliding?
Sailing?

5. logz.io | @logzio | @tomerlevy | @asafyigal
ELK – What is it good for?

6. logz.io | @logzio | @tomerlevy | @asafyigal
This is our office

7. logz.io | @logzio | @tomerlevy | @asafyigal
And this is the view from the office

8. logz.io | @logzio | @tomerlevy | @asafyigal
When should we leave
everything and go
kitesurfing?
This is our challenge

9. logz.io | @logzio | @tomerlevy | @asafyigal
Wind analytics

10. logz.io | @logzio | @tomerlevy | @asafyigal
Wind across locations

11. logz.io | @logzio | @tomerlevy | @asafyigal
Next steps

12. logz.io | @logzio | @tomerlevy | @asafyigal
Wind analytics — next items
• Alerts
• Wind forecast combined with real wind
• Wind predictions!

13. logz.io | @logzio | @tomerlevy | @asafyigal
Who are we?
• Logz.io Insights: Behavioural intelligence to pinpoint
what actually matters in logs
• ELK++ as a Service
– Infinitely scalable
– Secured
– Highly Available
– Additional Features (alerts, role-based access)

14. logz.io | @logzio | @tomerlevy | @asafyigal
ELK implementation —
creating the right architecture

15. logz.io | @logzio | @tomerlevy | @asafyigal
ELK implementation —
creating the right architecture
Curator
Curator
3x Master Nodes + 1 data

16. logz.io | @logzio | @tomerlevy | @asafyigal
ELK implementation —
creating the right architecture
Curator
3x Master Nodes + 1 data
Index Failures
Handler

17. logz.io | @logzio | @tomerlevy | @asafyigal
ELK basic implementation —
find the weak spots
AZ-1
AZ-2
ELBLB

18. logz.io | @logzio | @tomerlevy | @asafyigal
• Grok – parse logs to extract the relevant
fields…
• Try our blog for some help on grok/plugins etc’
• blog.logz.io
ELK basic implementation — configuration

19. logz.io | @logzio | @tomerlevy | @asafyigal
1. Use Elasticsearch AWS Plugin
2. EBS are challenging for big environment 
Use PIOPS if you can afford
3. Don’t run AWS cluster on the same AZ (but
don’t run them on different zones!) - use Shard
allocation awareness
4. S3 Snapshots are cool! Things tend to break…
Elasticsearch basic implementation —
configuration

20. logz.io | @logzio | @tomerlevy | @asafyigal
Let’s see a high-level view
of how we process logs
Demo

21. logz.io | @logzio | @tomerlevy | @asafyigal
Want to try our product?
• Email us with questions:
tomer@logz.io
asaf@logz.io
• Visit our website for more information!

22. logz.io | @logzio | @tomerlevy | @asafyigal
We’re hiring — big time!
• Elasticsearch experts
• Java developers
• Machine-learning experts
• See our job listings page