SlideShare a Scribd company logo
The ELK Stack @ Linko
Jilles van Gurp - Linko Inc.
Who is Jilles?
@jillesvangurp, www.jillesvangurp.com, and jillesvangurp on Github & just
about everything else.
Java (J)Ruby Python Javascript GEO
Server stuffreluctant Devops guy Software Architecture
Universities of Utrecht (NL), Blekinge (SE), and Groningen (NL)
GX Creative Online Development (NL)
Nokia Research (FI), Nokia/Here (DE)
Localstream (DE), Linko (DE).
Logging
Stuff runs
Produces errors, warnings, debug, telemetry,
analytics events, and other information
How to make sense of it?
Old school: Cat, grep, awk, cut, ….
Good luck with that on 200GB of unstructured
logs. Think lots of coffee breaks.
The fix: ELK
Or do the same stuff in Hadoop
Works great for structured data if you know
what you are looking for.
Requires a lot of infrastructure and hassle.
Not real-time, hard to explore data
I’m not a data scientist, are you?
The fix: ELK
ELK Stack?
Elasticsearch
Logstash
Kibana
ELK - Elasticsearch
Sharded, replicated, searchable, json document store.
Used by many big name services out there - Github,
Soundcloud, Foursquare, Xing, many others.
Full text search, geo spatial search, advanced search
ranking, suggestions, … much more. It’s awesome.
Nice HTTP API
Scaling Elasticsearch
1 node, 16GB, all of open streetmap in
geojson format (+ some other stuff) ->
reverse geocode in <100ms
There are people running ES with thousands
of nodes, trillions of documents, and
petabytes ...
Bottom line
Elasticsearch scales, probably way beyond
your needs
Log data is actually easy for elasticsearch
Elk - Logstash
Plumbing for your logs
Many different inputs for your logs
Filtering/parsing for your logs
Many outputs for your logs: for example redis,
elasticsearch, file,
ELK - Kibana
Highly configurable dashboard to slice and
dice your logstash logs in elasticsearch.
Real-time dashboards, easily configurable
ELK at Linko
Java Logback
NGINX
collectd
APP Servers
Linko Logstash - App Server (1)
input {
file {
type => "nginx_access"
path => ["/var/log/nginx/*.log"]
exclude => ["*.gz”, “error.*"]
discover_interval => 10
sincedb_path => "/opt/logstash/sincedb-
access-nginx"
}
}
filter {
grok {
type => "nginx_access"
patterns_dir => "/opt/logstash/patterns"
pattern =>
["%{NGINXACCESSWITHUPSTR}","%{NGINXACCESS}"]
}
date {
type => "nginx_access"
locale => "en"
match => [ "time_local" ,
"dd/MMM/YYYY:HH:mm:ss Z" ]
}
}
Grok pattern for NGINX
NGINXACCESSWITHUPSTR %{IPORHOST:remote_addr} - %{USERNAME:remote_user}
[%{HTTPDATE:time_local}] "%{WORD:method} %{URIPATHPARAM:request} %{GREEDYDATA:protocol}"
%{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent} %{QS:backend}
%{BASE16FLOAT:duration}
NGINXACCESS %{IPORHOST:remote_addr} - %{USERNAME:remote_user} [%{HTTPDATE:time_local}]
%{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}
Linko Logstash - App Server (2)
input {
file {
type => "backbone"
path => "/var/log/linko-
backbone/logstash/*.log"
codec => "json"
discover_interval => 10
sincedb_path => "/opt/logstash/sincedb-
access-backbone"
}
}
input {
collectd {
type => 'collectd'
}
}
output {
redis {
host => "192.168.1.13"
data_type => "list"
key => "logstash"
}
}
Linko Logstash - Elasticsearch
input {
redis {
host => "192.168.1.13"
# these settings should match the output
of the agent
data_type => "list"
key => "logstash"
# We use the 'json' codec here because we
expect to read
# json events from redis.
codec => json
}
}
output {
elasticsearch_http {
host => "192.168.1.13"
manage_template => true
template_overwrite => true
template =>
"/opt/logstash/index_template.json"
}
}
Experience - mostly good
Many moving parts - each with their odd
problems and issues
All parts are evolving. Prepare to upgrade.
Documentation is not great.
Finding out the hard way ...
Rolling restarts with elasticsearch
Configuring caching because of OOM’s
Clicking together dashboards in Kibana
Don’t restart cluster nodes blindly
Beware: Split brain
Default ES config is not appropriate for
production
Gotchas
Kibana needs to talk to ES, but you don’t want
that exposed to the world.
ES Fielddata cache is unrestricted, by default
Elasticsearch_http can fail silently, if
misconfigured.
If you use file input, be sure to set the sincedb
Getting started
Download es & logstash to your laptop.
Simply run ES as is; worry about config later
Follow logstash cookbook to get started
Setup some simple inputs
Use elasticsearch_http, not elasticsearch output
Install kibana plugin in es
Open your browser
After getting started
RTFM, play, explore, mess up, google, …
Configure ES properly
Setup nginx/apache to proxy
Think about retention policies
...
Links
http://www.elasticsearch.org/
http://linko.io
https://groups.google.com/forum/?fromgroups
=#!forum/elasticsearch
http://www.jillesvangurp.com
Thanks!
@jillesvangurp, @linkoapp

More Related Content

What's hot

Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
Amr Alaa Yassen
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack Introduction
Vikram Shinde
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK Stack
Rohit Sharma
 
ELK introduction
ELK introductionELK introduction
ELK introduction
Waldemar Neto
 
Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...
Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...
Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...
ForgeRock
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
Rich Lee
 
Elk devops
Elk devopsElk devops
Elk devops
Ideato
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
Vikrant Chauhan
 
RocksDB Performance and Reliability Practices
RocksDB Performance and Reliability PracticesRocksDB Performance and Reliability Practices
RocksDB Performance and Reliability Practices
Yoshinori Matsunobu
 
Introduction to Elasticsearch with basics of Lucene
Introduction to Elasticsearch with basics of LuceneIntroduction to Elasticsearch with basics of Lucene
Introduction to Elasticsearch with basics of Lucene
Rahul Jain
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
Joe Ryan
 
Introduction to Elasticsearch
Introduction to ElasticsearchIntroduction to Elasticsearch
Introduction to Elasticsearch
Ruslan Zavacky
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELK
Geert Pante
 
Attack monitoring using ElasticSearch Logstash and Kibana
Attack monitoring using ElasticSearch Logstash and KibanaAttack monitoring using ElasticSearch Logstash and Kibana
Attack monitoring using ElasticSearch Logstash and Kibana
Prajal Kulkarni
 
Introduction to elasticsearch
Introduction to elasticsearchIntroduction to elasticsearch
Introduction to elasticsearch
hypto
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
Rushika Shah
 
Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]
Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]
Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]
MongoDB
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Edureka!
 
elk_stack_alexander_szalonnas
elk_stack_alexander_szalonnaselk_stack_alexander_szalonnas
elk_stack_alexander_szalonnas
Alexander Szalonnas
 
Introduction to Elasticsearch
Introduction to ElasticsearchIntroduction to Elasticsearch
Introduction to Elasticsearch
Ismaeel Enjreny
 

What's hot (20)

Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack Introduction
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK Stack
 
ELK introduction
ELK introductionELK introduction
ELK introduction
 
Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...
Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...
Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit ...
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
 
Elk devops
Elk devopsElk devops
Elk devops
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
 
RocksDB Performance and Reliability Practices
RocksDB Performance and Reliability PracticesRocksDB Performance and Reliability Practices
RocksDB Performance and Reliability Practices
 
Introduction to Elasticsearch with basics of Lucene
Introduction to Elasticsearch with basics of LuceneIntroduction to Elasticsearch with basics of Lucene
Introduction to Elasticsearch with basics of Lucene
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
 
Introduction to Elasticsearch
Introduction to ElasticsearchIntroduction to Elasticsearch
Introduction to Elasticsearch
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELK
 
Attack monitoring using ElasticSearch Logstash and Kibana
Attack monitoring using ElasticSearch Logstash and KibanaAttack monitoring using ElasticSearch Logstash and Kibana
Attack monitoring using ElasticSearch Logstash and Kibana
 
Introduction to elasticsearch
Introduction to elasticsearchIntroduction to elasticsearch
Introduction to elasticsearch
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
 
Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]
Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]
Naver속도의, 속도에 의한, 속도를 위한 몽고DB (네이버 컨텐츠검색과 몽고DB) [Naver]
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
 
elk_stack_alexander_szalonnas
elk_stack_alexander_szalonnaselk_stack_alexander_szalonnas
elk_stack_alexander_szalonnas
 
Introduction to Elasticsearch
Introduction to ElasticsearchIntroduction to Elasticsearch
Introduction to Elasticsearch
 

Viewers also liked

Logstash
LogstashLogstash
Logstash
琛琳 饶
 
Logging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & KibanaLogging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & Kibana
Amazee Labs
 
Elk ruminating on logs
Elk ruminating on logsElk ruminating on logs
Elk ruminating on logs
Mathew Beane
 
Consumer Driven Contracts and Your Microservice Architecture
Consumer Driven Contracts and Your Microservice ArchitectureConsumer Driven Contracts and Your Microservice Architecture
Consumer Driven Contracts and Your Microservice Architecture
Marcin Grzejszczak
 
How ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps lifeHow ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps life
琛琳 饶
 
Webinar usando graylog para la gestión centralizada de logs
Webinar usando graylog para la gestión centralizada de logsWebinar usando graylog para la gestión centralizada de logs
Webinar usando graylog para la gestión centralizada de logs
atSistemas
 
Monitoring with Graylog - a modern approach to monitoring?
Monitoring with Graylog - a modern approach to monitoring?Monitoring with Graylog - a modern approach to monitoring?
Monitoring with Graylog - a modern approach to monitoring?
inovex GmbH
 
Advanced troubleshooting linux performance
Advanced troubleshooting linux performanceAdvanced troubleshooting linux performance
Advanced troubleshooting linux performance
Forthscale
 
Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)
Reshmi Krishna
 

Viewers also liked (9)

Logstash
LogstashLogstash
Logstash
 
Logging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & KibanaLogging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & Kibana
 
Elk ruminating on logs
Elk ruminating on logsElk ruminating on logs
Elk ruminating on logs
 
Consumer Driven Contracts and Your Microservice Architecture
Consumer Driven Contracts and Your Microservice ArchitectureConsumer Driven Contracts and Your Microservice Architecture
Consumer Driven Contracts and Your Microservice Architecture
 
How ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps lifeHow ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps life
 
Webinar usando graylog para la gestión centralizada de logs
Webinar usando graylog para la gestión centralizada de logsWebinar usando graylog para la gestión centralizada de logs
Webinar usando graylog para la gestión centralizada de logs
 
Monitoring with Graylog - a modern approach to monitoring?
Monitoring with Graylog - a modern approach to monitoring?Monitoring with Graylog - a modern approach to monitoring?
Monitoring with Graylog - a modern approach to monitoring?
 
Advanced troubleshooting linux performance
Advanced troubleshooting linux performanceAdvanced troubleshooting linux performance
Advanced troubleshooting linux performance
 
Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)
 

Similar to Elk stack

Logstash for SEO: come monitorare i Log del Web Server in realtime
Logstash for SEO: come monitorare i Log del Web Server in realtimeLogstash for SEO: come monitorare i Log del Web Server in realtime
Logstash for SEO: come monitorare i Log del Web Server in realtime
Andrea Cardinale
 
dojo.Patterns
dojo.Patternsdojo.Patterns
dojo.Patterns
Peter Higgins
 
(Fios#02) 2. elk 포렌식 분석
(Fios#02) 2. elk 포렌식 분석(Fios#02) 2. elk 포렌식 분석
(Fios#02) 2. elk 포렌식 분석
INSIGHT FORENSIC
 
Anwendungsfaelle für Elasticsearch
Anwendungsfaelle für ElasticsearchAnwendungsfaelle für Elasticsearch
Anwendungsfaelle für Elasticsearch
Florian Hopf
 
Go react codelab
Go react codelabGo react codelab
GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes with ...
GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes  with ...GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes  with ...
GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes with ...
KAI CHU CHUNG
 
Trimming The Cruft
Trimming The CruftTrimming The Cruft
Trimming The Cruft
Peter Higgins
 
1.6 米嘉 gobuildweb
1.6 米嘉 gobuildweb1.6 米嘉 gobuildweb
1.6 米嘉 gobuildweb
Leo Zhou
 
ELK: a log management framework
ELK: a log management frameworkELK: a log management framework
ELK: a log management framework
Giovanni Bechis
 
Cross Domain Web
Mashups with JQuery and Google App Engine
Cross Domain Web
Mashups with JQuery and Google App EngineCross Domain Web
Mashups with JQuery and Google App Engine
Cross Domain Web
Mashups with JQuery and Google App Engine
Andy McKay
 
Grokking Grok: Monitorama PDX 2015
Grokking Grok: Monitorama PDX 2015Grokking Grok: Monitorama PDX 2015
Grokking Grok: Monitorama PDX 2015
GregMefford
 
The Road To Reactive with RxJava JEEConf 2016
The Road To Reactive with RxJava JEEConf 2016The Road To Reactive with RxJava JEEConf 2016
The Road To Reactive with RxJava JEEConf 2016
Frank Lyaruu
 
Elk stack @inbot
Elk stack @inbotElk stack @inbot
Elk stack @inbot
Jilles van Gurp
 
Appsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaolaAppsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaola
drewz lin
 
Grooscript greach 2015
Grooscript greach 2015Grooscript greach 2015
Grooscript greach 2015
Jorge Franco Leza
 
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.ioReal-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Rick Copeland
 
Original slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talkOriginal slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talk
Aarti Parikh
 
Node azure
Node azureNode azure
Node azure
Emanuele DelBono
 
Smarter internet of things with stream and event processing virtual io_t_meet...
Smarter internet of things with stream and event processing virtual io_t_meet...Smarter internet of things with stream and event processing virtual io_t_meet...
Smarter internet of things with stream and event processing virtual io_t_meet...
Istvan Rath
 
Accelerated Adoption: HTML5 and CSS3 for ASP.NET Developers
Accelerated Adoption: HTML5 and CSS3 for ASP.NET DevelopersAccelerated Adoption: HTML5 and CSS3 for ASP.NET Developers
Accelerated Adoption: HTML5 and CSS3 for ASP.NET Developers
Todd Anglin
 

Similar to Elk stack (20)

Logstash for SEO: come monitorare i Log del Web Server in realtime
Logstash for SEO: come monitorare i Log del Web Server in realtimeLogstash for SEO: come monitorare i Log del Web Server in realtime
Logstash for SEO: come monitorare i Log del Web Server in realtime
 
dojo.Patterns
dojo.Patternsdojo.Patterns
dojo.Patterns
 
(Fios#02) 2. elk 포렌식 분석
(Fios#02) 2. elk 포렌식 분석(Fios#02) 2. elk 포렌식 분석
(Fios#02) 2. elk 포렌식 분석
 
Anwendungsfaelle für Elasticsearch
Anwendungsfaelle für ElasticsearchAnwendungsfaelle für Elasticsearch
Anwendungsfaelle für Elasticsearch
 
Go react codelab
Go react codelabGo react codelab
Go react codelab
 
GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes with ...
GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes  with ...GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes  with ...
GDG Cloud Taipei meetup #50 - Build go kit microservices at kubernetes with ...
 
Trimming The Cruft
Trimming The CruftTrimming The Cruft
Trimming The Cruft
 
1.6 米嘉 gobuildweb
1.6 米嘉 gobuildweb1.6 米嘉 gobuildweb
1.6 米嘉 gobuildweb
 
ELK: a log management framework
ELK: a log management frameworkELK: a log management framework
ELK: a log management framework
 
Cross Domain Web
Mashups with JQuery and Google App Engine
Cross Domain Web
Mashups with JQuery and Google App EngineCross Domain Web
Mashups with JQuery and Google App Engine
Cross Domain Web
Mashups with JQuery and Google App Engine
 
Grokking Grok: Monitorama PDX 2015
Grokking Grok: Monitorama PDX 2015Grokking Grok: Monitorama PDX 2015
Grokking Grok: Monitorama PDX 2015
 
The Road To Reactive with RxJava JEEConf 2016
The Road To Reactive with RxJava JEEConf 2016The Road To Reactive with RxJava JEEConf 2016
The Road To Reactive with RxJava JEEConf 2016
 
Elk stack @inbot
Elk stack @inbotElk stack @inbot
Elk stack @inbot
 
Appsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaolaAppsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaola
 
Grooscript greach 2015
Grooscript greach 2015Grooscript greach 2015
Grooscript greach 2015
 
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.ioReal-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
 
Original slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talkOriginal slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talk
 
Node azure
Node azureNode azure
Node azure
 
Smarter internet of things with stream and event processing virtual io_t_meet...
Smarter internet of things with stream and event processing virtual io_t_meet...Smarter internet of things with stream and event processing virtual io_t_meet...
Smarter internet of things with stream and event processing virtual io_t_meet...
 
Accelerated Adoption: HTML5 and CSS3 for ASP.NET Developers
Accelerated Adoption: HTML5 and CSS3 for ASP.NET DevelopersAccelerated Adoption: HTML5 and CSS3 for ASP.NET Developers
Accelerated Adoption: HTML5 and CSS3 for ASP.NET Developers
 

Recently uploaded

Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Peter Caitens
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
Peter Muessig
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
Marcin Chrost
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
Tier1 app
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Peter Muessig
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
OnePlan Solutions
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
kgyxske
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
Jhone kinadey
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
sandeepmenon62
 
Kubernetes at Scale: Going Multi-Cluster with Istio
Kubernetes at Scale:  Going Multi-Cluster  with IstioKubernetes at Scale:  Going Multi-Cluster  with Istio
Kubernetes at Scale: Going Multi-Cluster with Istio
Severalnines
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 

Recently uploaded (20)

Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
 
Kubernetes at Scale: Going Multi-Cluster with Istio
Kubernetes at Scale:  Going Multi-Cluster  with IstioKubernetes at Scale:  Going Multi-Cluster  with Istio
Kubernetes at Scale: Going Multi-Cluster with Istio
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 

Elk stack

  • 1. The ELK Stack @ Linko Jilles van Gurp - Linko Inc.
  • 2.
  • 3. Who is Jilles? @jillesvangurp, www.jillesvangurp.com, and jillesvangurp on Github & just about everything else. Java (J)Ruby Python Javascript GEO Server stuffreluctant Devops guy Software Architecture Universities of Utrecht (NL), Blekinge (SE), and Groningen (NL) GX Creative Online Development (NL) Nokia Research (FI), Nokia/Here (DE) Localstream (DE), Linko (DE).
  • 4. Logging Stuff runs Produces errors, warnings, debug, telemetry, analytics events, and other information How to make sense of it?
  • 5. Old school: Cat, grep, awk, cut, …. Good luck with that on 200GB of unstructured logs. Think lots of coffee breaks. The fix: ELK
  • 6. Or do the same stuff in Hadoop Works great for structured data if you know what you are looking for. Requires a lot of infrastructure and hassle. Not real-time, hard to explore data I’m not a data scientist, are you? The fix: ELK
  • 8. ELK - Elasticsearch Sharded, replicated, searchable, json document store. Used by many big name services out there - Github, Soundcloud, Foursquare, Xing, many others. Full text search, geo spatial search, advanced search ranking, suggestions, … much more. It’s awesome. Nice HTTP API
  • 9.
  • 10. Scaling Elasticsearch 1 node, 16GB, all of open streetmap in geojson format (+ some other stuff) -> reverse geocode in <100ms There are people running ES with thousands of nodes, trillions of documents, and petabytes ...
  • 11. Bottom line Elasticsearch scales, probably way beyond your needs Log data is actually easy for elasticsearch
  • 12. Elk - Logstash Plumbing for your logs Many different inputs for your logs Filtering/parsing for your logs Many outputs for your logs: for example redis, elasticsearch, file,
  • 13.
  • 14. ELK - Kibana Highly configurable dashboard to slice and dice your logstash logs in elasticsearch. Real-time dashboards, easily configurable
  • 15.
  • 16.
  • 17. ELK at Linko Java Logback NGINX collectd APP Servers
  • 18. Linko Logstash - App Server (1) input { file { type => "nginx_access" path => ["/var/log/nginx/*.log"] exclude => ["*.gz”, “error.*"] discover_interval => 10 sincedb_path => "/opt/logstash/sincedb- access-nginx" } } filter { grok { type => "nginx_access" patterns_dir => "/opt/logstash/patterns" pattern => ["%{NGINXACCESSWITHUPSTR}","%{NGINXACCESS}"] } date { type => "nginx_access" locale => "en" match => [ "time_local" , "dd/MMM/YYYY:HH:mm:ss Z" ] } }
  • 19. Grok pattern for NGINX NGINXACCESSWITHUPSTR %{IPORHOST:remote_addr} - %{USERNAME:remote_user} [%{HTTPDATE:time_local}] "%{WORD:method} %{URIPATHPARAM:request} %{GREEDYDATA:protocol}" %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent} %{QS:backend} %{BASE16FLOAT:duration} NGINXACCESS %{IPORHOST:remote_addr} - %{USERNAME:remote_user} [%{HTTPDATE:time_local}] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}
  • 20. Linko Logstash - App Server (2) input { file { type => "backbone" path => "/var/log/linko- backbone/logstash/*.log" codec => "json" discover_interval => 10 sincedb_path => "/opt/logstash/sincedb- access-backbone" } } input { collectd { type => 'collectd' } } output { redis { host => "192.168.1.13" data_type => "list" key => "logstash" } }
  • 21. Linko Logstash - Elasticsearch input { redis { host => "192.168.1.13" # these settings should match the output of the agent data_type => "list" key => "logstash" # We use the 'json' codec here because we expect to read # json events from redis. codec => json } } output { elasticsearch_http { host => "192.168.1.13" manage_template => true template_overwrite => true template => "/opt/logstash/index_template.json" } }
  • 22. Experience - mostly good Many moving parts - each with their odd problems and issues All parts are evolving. Prepare to upgrade. Documentation is not great.
  • 23. Finding out the hard way ... Rolling restarts with elasticsearch Configuring caching because of OOM’s Clicking together dashboards in Kibana Don’t restart cluster nodes blindly Beware: Split brain Default ES config is not appropriate for production
  • 24. Gotchas Kibana needs to talk to ES, but you don’t want that exposed to the world. ES Fielddata cache is unrestricted, by default Elasticsearch_http can fail silently, if misconfigured. If you use file input, be sure to set the sincedb
  • 25. Getting started Download es & logstash to your laptop. Simply run ES as is; worry about config later Follow logstash cookbook to get started Setup some simple inputs Use elasticsearch_http, not elasticsearch output Install kibana plugin in es Open your browser
  • 26. After getting started RTFM, play, explore, mess up, google, … Configure ES properly Setup nginx/apache to proxy Think about retention policies ...