Log analysis OpenSource con Logstash, Elasticsearch e Kibana

  • 4,175 views
Uploaded on

Log analysis Open Source con Logstash, Elasticsearch e Kibana. …

Log analysis Open Source con Logstash, Elasticsearch e Kibana.
A cosa serve la log analysis? Panoramica sulle possibilita' offerte da Logstash, Elasticsearch e Kibana per la gestione centralizzata open source di log.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,175
On Slideshare
0
From Embeds
0
Number of Embeds
7

Actions

Shares
Downloads
35
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. #servertraining Log Analysis Open Source con Logstash Elasticsearch & Kibana www.servermanaged.it twitter.com/servermanagedit
  • 2. #servertraining Slides a cura di Valentino Gagliardi Technical Manager at ServerManaged.it Devop & Sysadmin vecchia scuola, consulente informatico per small/medium business, cloud, hosting operations. www.servermanaged.it twitter.com/servermanagedit
  • 3. #servertraining Sommario Log analysis, a cosa serve? ●In principio era... ●Cos'e' Logstash ●Cos'e' Elasticsearch ●Cos'e' Kibana ●The big picture, un setup tipico ●Ma Splunk? E Loggly? ●Logstash, vantaggi ● www.servermanaged.it twitter.com/servermanagedit
  • 4. #servertraining Log analysis: “is an art and science seeking to make sense out of computergenerated records” www.servermanaged.it twitter.com/servermanagedit
  • 5. #servertraining Tradotto: dare un senso ad una montagna di log provenienti da server, routers, ecc www.servermanaged.it twitter.com/servermanagedit
  • 6. #servertraining Log: I registri di un sistema. Se c'e' un problema sul server Y c'e' anche una traccia nei log. www.servermanaged.it twitter.com/servermanagedit
  • 7. #servertraining Log analysis per: - tracciare i problemi - prevenire incidenti di sicurezza - ricostruirli se avvengono www.servermanaged.it twitter.com/servermanagedit
  • 8. #servertraining In principio era tailf ... www.servermanaged.it twitter.com/servermanagedit
  • 9. #servertraining # tailf /var/log/secure www.servermanaged.it twitter.com/servermanagedit
  • 10. #servertraining # tailf /var/log/secure # tailf /var/log/messages www.servermanaged.it twitter.com/servermanagedit
  • 11. #servertraining # tailf /var/log/secure # tailf /var/log/messages # multitail /var/log/httpd/error.log /var/log/httpd/access.log www.servermanaged.it twitter.com/servermanagedit
  • 12. #servertraining Oggi e' data visualization www.servermanaged.it twitter.com/servermanagedit
  • 13. #servertraining www.servermanaged.it twitter.com/servermanagedit
  • 14. #servertraining Cos'e' Logstash? “Logstash helps you take logs and other event data from your systems and store them in a central place. “ www.servermanaged.it twitter.com/servermanagedit
  • 15. #servertraining Logstash: trasforma qualsiasi fonte di eventi e log in qualcosa di digeribile e processabile www.servermanaged.it twitter.com/servermanagedit
  • 16. #servertraining Logstash: 36 inputs (and growing) 14 codecs (and growing) 40 filtri (and growing) 50 outputs (and growing) www.servermanaged.it twitter.com/servermanagedit
  • 17. #servertraining ## Una configurazione minimale di Logstash input { file { type => "linux-syslog" path => ["/var/log/*.log"] exclude => [ "*.gz" ] } } output { redis { host => "127.0.0.1" data_type => "list" key => "syslog" } } ## www.servermanaged.it twitter.com/servermanagedit
  • 18. #servertraining Cos'e' Redis? “Redis is an open source, BSD licensed, advanced key-value store.“ www.servermanaged.it twitter.com/servermanagedit
  • 19. #servertraining Redis: in un sistema di logging centralizzato puo' essere usato come buffer per i log www.servermanaged.it twitter.com/servermanagedit
  • 20. #servertraining Cos'e' Elasticsearch? “flexible and powerful open source, distributed real-time search and analytics engine for the cloud“ www.servermanaged.it twitter.com/servermanagedit
  • 21. #servertraining Elasticsearch: in un sistema di logging centralizzato puo' essere usato come output per indicizzare i log www.servermanaged.it twitter.com/servermanagedit
  • 22. #servertraining Cos'e' Kibana? “Make Sense of your Data“ www.servermanaged.it twitter.com/servermanagedit
  • 23. #servertraining Kibana: una dashboard per estrarre i dati da Elasticsearch www.servermanaged.it twitter.com/servermanagedit
  • 24. #servertraining www.servermanaged.it twitter.com/servermanagedit
  • 25. #servertraining Big picture: logging centralizzato www.servermanaged.it twitter.com/servermanagedit
  • 26. #servertraining Un setup tipico per il logging centralizzato con Rsyslog, Logstash, Redis, Elasticsearch e Kibana. www.servermanaged.it twitter.com/servermanagedit
  • 27. #servertraining server Server di Logging Centralizzato server server rsyslog server www.servermanaged.it twitter.com/servermanagedit
  • 28. #servertraining server Server di Logging Centralizzato server server server rsyslog server server www.servermanaged.it twitter.com/servermanagedit
  • 29. #servertraining server Server di Logging Centralizzato server server server rsyslog server server www.servermanaged.it twitter.com/servermanagedit
  • 30. #servertraining server Server di Logging Centralizzato server server server rsyslog server server www.servermanaged.it twitter.com/servermanagedit
  • 31. #servertraining server Server di Logging Centralizzato server server server rsyslog server server www.servermanaged.it twitter.com/servermanagedit
  • 32. #servertraining server Server di Logging Centralizzato server server server rsyslog server server www.servermanaged.it twitter.com/servermanagedit
  • 33. #servertraining Ok tutto bello. “Ma cosa te ne fai di questi grafici?” (tratto da una domanda vera) www.servermanaged.it twitter.com/servermanagedit
  • 34. #servertraining Centralizzare i log: - visibilita' dei trend - visibilita' dei problemi - analisi di sicurezza www.servermanaged.it twitter.com/servermanagedit
  • 35. #servertraining Caso di studio. Mitigazione di una serie di potenti attacchi bruteforce www.servermanaged.it twitter.com/servermanagedit
  • 36. #servertraining Visualizzare le conseguenze di un attacco, anticipare il trend e mitigare la minaccia. In rosso: attacco bruteforce massivo su siti web Joomla In verde: mitigazione dell'attacco. Le richieste anomale vengono respinte www.servermanaged.it twitter.com/servermanagedit
  • 37. #servertraining Non solo Logstash: Le alternative costose al logging centralizzato open source. www.servermanaged.it twitter.com/servermanagedit
  • 38. #servertraining Splunk> Grab a 20GB license for 12187631461319$/month (gratuito fino a 500MB/giorno) www.servermanaged.it twitter.com/servermanagedit
  • 39. #servertraining Splunk> www.servermanaged.it twitter.com/servermanagedit
  • 40. #servertraining Loggly 10GB di log al giorno con una ritenzione di 90 giorni costano circa 1482 dollari al mese. www.servermanaged.it twitter.com/servermanagedit
  • 41. #servertraining Ogni prodotto ha pro e contro. Splunk e Loggly: compliance e immediatezza. Logstash: per i nerd. www.servermanaged.it twitter.com/servermanagedit
  • 42. #servertraining Logstash: vantaggi - open source - gratuito - una folta community - in continuo sviluppo www.servermanaged.it twitter.com/servermanagedit
  • 43. #servertraining KEEP CALM AND LOGSTASH www.servermanaged.it twitter.com/servermanagedit
  • 44. #servertraining http://www.logstash.net http://www.redis.io http://www.elasticsearch.org www.servermanaged.it twitter.com/servermanagedit
  • 45. #servertraining Slides a cura di Valentino Gagliardi Technical Manager at ServerManaged.it Devop & Sysadmin vecchia scuola, consulente informatico per small/medium business, cloud, hosting operations. (Vieni a trovarmi su Google+, LinkedIn e Twitter) Immagine di sfondo: http://medialoot.com/item/free-dark-noise-backgrounds www.servermanaged.it twitter.com/servermanagedit