SlideShare a Scribd company logo

ELK Stack

•
•
Eberhard Wolff
Eberhard Wolff

Short presentation about the ELK stack (Elasticsearch, Logstash, Kibana) running on top of Docker / Vagrant.

1 of 19
Download to read offline
ELK
Eberhard Wolff - @ewolff Docker •  No true virtualization •  Linux Containers (lxc) •  i.e. shared kernel •  i.e. separate file systems
Eberhard Wolff - @ewolff Docker File Systems •  Read only base images •  +read/write image •  Can be stacked
Eberhard Wolff - @ewolff Docker •  Linux only •  So: Run it in VM •  Controlled by Vagrant
Docker Demo
Eberhard Wolff - @ewolff Communication Between Docker Container •  Via ports •  Via data volumes
Docker is a Component System
Eberhard Wolff - @ewolff Example •  Log file analysis •  ELK Stack •  Elasticsearch: Store •  Logstash: Parse •  Kibana: Web based GUI •  Install three servers •  Automate installation ... L
Eberhard Wolff - @ewolff Dockerized ELK Stack •  1 Vagrantfile (25 lines) •  6 Dockerfiles (total 28 lines) •  1 HTML file (11 lines) •  1 Config (Logstash) (29 lines) •  Total: 93 lines, 9 files •  Including one RegExp L
Eberhard Wolff - @ewolff Vagrant VM Container user-registration 8080 8081 Port 8081 Container elasticsearch Container logstash 9200 9200 Port 9200 9200 link Container kibana Port 8080 8080 8080 Browser Volume /log
Demo ELK Stack
Eberhard Wolff - @ewolff date time level 1988-10-12 19:42:07.350 INFO 683 [http-nio-8080-exec-6] --- c.e.u.service.RegistrationService : email=eberhard.wolff@gmail.com geloesc pid thread class logmessage
Eberhard Wolff - @ewolff Scaling Elasticsearch •  More servers •  Shards •  Easy to set up •  Index per day or week or month •  Smaller dataset
Eberhard Wolff - @ewolff Ideas about Logstash •  No parsing •  Send JSON
Eberhard Wolff - @ewolff Break Logstash apart •  Logstash does parsing and communication •  Not too sensible in a distributed sytem •  Separate process per responsibility
Eberhard Wolff - @ewolff Shipper •  Ship data from logging system •  Logstash •  Beaver https://github.com/josegonzalez/ beaver •  Woodchuck https://github.com/danryan/ woodchuck
Eberhard Wolff - @ewolff Broker •  Buffer data for later processing •  Redis •  In-Memory NoSQL database •  Small •  Fast •  ...and then Parse (Logstash)
Eberhard Wolff - @ewolff Links •  https://github.com/ewolff/user- registration
Eberhard Wolff - @ewolff Thank You!! eberhard.wolff@gmail.com @ewolff

Recommended

ELK Stack
ELK StackELK Stack
ELK Stack
Phuc Nguyen
 
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashKeeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Amazon Web Services
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
Edureka!
 
Elk
Elk Elk
Elk
Caleb Wang
 
The Elastic ELK Stack
The Elastic ELK StackThe Elastic ELK Stack
The Elastic ELK Stack
enterprisesearchmeetup
 
Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
Amr Alaa Yassen
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack Introduction
Vikram Shinde
 
ELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log ManagementELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log Management
El Mahdi Benzekri
 

Recommended

ELK Stack
ELK StackELK Stack
ELK Stack
Phuc Nguyen
 
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashKeeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Amazon Web Services
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
Edureka!
 
Elk
Elk Elk
Elk
Caleb Wang
 
The Elastic ELK Stack
The Elastic ELK StackThe Elastic ELK Stack
The Elastic ELK Stack
enterprisesearchmeetup
 
Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
Amr Alaa Yassen
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack Introduction
Vikram Shinde
 
ELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log ManagementELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log Management
El Mahdi Benzekri
 
Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & Kibana
SpringPeople
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
Joe Ryan
 
ELK introduction
ELK introductionELK introduction
ELK introduction
Waldemar Neto
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
Vikrant Chauhan
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELK
Geert Pante
 
Introducing ELK
Introducing ELKIntroducing ELK
Introducing ELKAllBits BVBA (freelancer)
 
Distributed tracing 101
Distributed tracing 101Distributed tracing 101
Distributed tracing 101
Itiel Shwartz
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK Stack
Rohit Sharma
 
Cloud Monitoring tool Grafana
Cloud Monitoring tool Grafana Cloud Monitoring tool Grafana
Cloud Monitoring tool Grafana
Dhrubaji Mandal ♛
 
Elk - An introduction
Elk - An introductionElk - An introduction
Elk - An introduction
Hossein Shemshadi
 
Introduction to ELK
Introduction to ELKIntroduction to ELK
Introduction to ELKHarshakumar Ummerpillai
 
Introduction To Kibana
Introduction To KibanaIntroduction To Kibana
Introduction To Kibana
Jen Stirrup
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
Rich Lee
 
Stephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large ScaleStephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large Scale
Ververica
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observability
Elasticsearch
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to Kibana
Vineet .
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
Jeff Holoman
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
Rushika Shah
 
ElasticSearch at berlinbuzzwords 2010
ElasticSearch at berlinbuzzwords 2010ElasticSearch at berlinbuzzwords 2010
ElasticSearch at berlinbuzzwords 2010
Elasticsearch
 
Kafka Retry and DLQ
Kafka Retry and DLQKafka Retry and DLQ
Kafka Retry and DLQ
George Teo
 
Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is Beautiful
Eberhard Wolff
 
Java Architectures - a New Hope
Java Architectures - a New HopeJava Architectures - a New Hope
Java Architectures - a New Hope
Eberhard Wolff
 

More Related Content

What's hot

Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & Kibana
SpringPeople
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
Joe Ryan
 
ELK introduction
ELK introductionELK introduction
ELK introduction
Waldemar Neto
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
Vikrant Chauhan
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELK
Geert Pante
 
Distributed tracing 101
Distributed tracing 101Distributed tracing 101
Distributed tracing 101
Itiel Shwartz
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK Stack
Rohit Sharma
 
Cloud Monitoring tool Grafana
Cloud Monitoring tool Grafana Cloud Monitoring tool Grafana
Cloud Monitoring tool Grafana
Dhrubaji Mandal ♛
 
Elk - An introduction
Elk - An introductionElk - An introduction
Elk - An introduction
Hossein Shemshadi
 
Introduction To Kibana
Introduction To KibanaIntroduction To Kibana
Introduction To Kibana
Jen Stirrup
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
Rich Lee
 
Stephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large ScaleStephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large Scale
Ververica
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observability
Elasticsearch
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to Kibana
Vineet .
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
Jeff Holoman
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
Rushika Shah
 
ElasticSearch at berlinbuzzwords 2010
ElasticSearch at berlinbuzzwords 2010ElasticSearch at berlinbuzzwords 2010
ElasticSearch at berlinbuzzwords 2010
Elasticsearch
 
Kafka Retry and DLQ
Kafka Retry and DLQKafka Retry and DLQ
Kafka Retry and DLQ
George Teo
 

What's hot (20)

Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & Kibana
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
 
ELK introduction
ELK introductionELK introduction
ELK introduction
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELK
 
Introducing ELK
Introducing ELKIntroducing ELK
Introducing ELK
 
Distributed tracing 101
Distributed tracing 101Distributed tracing 101
Distributed tracing 101
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK Stack
 
Cloud Monitoring tool Grafana
Cloud Monitoring tool Grafana Cloud Monitoring tool Grafana
Cloud Monitoring tool Grafana
 
Elk - An introduction
Elk - An introductionElk - An introduction
Elk - An introduction
 
Introduction to ELK
Introduction to ELKIntroduction to ELK
Introduction to ELK
 
Introduction To Kibana
Introduction To KibanaIntroduction To Kibana
Introduction To Kibana
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
 
Stephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large ScaleStephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large Scale
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observability
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to Kibana
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
 
ElasticSearch at berlinbuzzwords 2010
ElasticSearch at berlinbuzzwords 2010ElasticSearch at berlinbuzzwords 2010
ElasticSearch at berlinbuzzwords 2010
 
Kafka Retry and DLQ
Kafka Retry and DLQKafka Retry and DLQ
Kafka Retry and DLQ
 

Similar to ELK Stack

Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is Beautiful
Eberhard Wolff
 
Java Architectures - a New Hope
Java Architectures - a New HopeJava Architectures - a New Hope
Java Architectures - a New Hope
Eberhard Wolff
 
NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?
Eberhard Wolff
 
Micro Service – The New Architecture Paradigm
Micro Service – The New Architecture ParadigmMicro Service – The New Architecture Paradigm
Micro Service – The New Architecture Paradigm
Eberhard Wolff
 
High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...
Eberhard Wolff
 
Docker Dojo
Docker DojoDocker Dojo
Docker Dojo
Hugo González Labrador
 
Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?
Eberhard Wolff
 
DockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and DockerDockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and Docker
Taylor Brown
 
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Docker, Inc.
 
Java Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short VersionJava Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short Version
Eberhard Wolff
 
Java Application Servers Are Dead!
Java Application Servers Are Dead!Java Application Servers Are Dead!
Java Application Servers Are Dead!
Eberhard Wolff
 
Microservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring CloudMicroservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring Cloud
Eberhard Wolff
 
Openstack Cactus Survey
Openstack Cactus SurveyOpenstack Cactus Survey
Openstack Cactus SurveyPjack Chen
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Alan Forbes
 
Spring Boot
Spring BootSpring Boot
Spring Boot
gedoplan
 
Docker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker NetworkingDocker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker Networking
Docker, Inc.
 
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anyninesCloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
anynines GmbH
 
Magnum Networking Update
Magnum Networking UpdateMagnum Networking Update
Magnum Networking Update
Daneyon Hansen
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with Docker
Ravindu Fernando
 
Amazon Elastic Beanstalk
Amazon Elastic BeanstalkAmazon Elastic Beanstalk
Amazon Elastic BeanstalkEberhard Wolff
 

Similar to ELK Stack (20)

Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is Beautiful
 
Java Architectures - a New Hope
Java Architectures - a New HopeJava Architectures - a New Hope
Java Architectures - a New Hope
 
NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?
 
Micro Service – The New Architecture Paradigm
Micro Service – The New Architecture ParadigmMicro Service – The New Architecture Paradigm
Micro Service – The New Architecture Paradigm
 
High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...
 
Docker Dojo
Docker DojoDocker Dojo
Docker Dojo
 
Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?
 
DockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and DockerDockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and Docker
 
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
 
Java Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short VersionJava Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short Version
 
Java Application Servers Are Dead!
Java Application Servers Are Dead!Java Application Servers Are Dead!
Java Application Servers Are Dead!
 
Microservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring CloudMicroservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring Cloud
 
Openstack Cactus Survey
Openstack Cactus SurveyOpenstack Cactus Survey
Openstack Cactus Survey
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Spring Boot
Spring BootSpring Boot
Spring Boot
 
Docker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker NetworkingDocker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker Networking
 
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anyninesCloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
 
Magnum Networking Update
Magnum Networking UpdateMagnum Networking Update
Magnum Networking Update
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with Docker
 
Amazon Elastic Beanstalk
Amazon Elastic BeanstalkAmazon Elastic Beanstalk
Amazon Elastic Beanstalk
 

More from Eberhard Wolff

Architectures and Alternatives
Architectures and AlternativesArchitectures and Alternatives
Architectures and Alternatives
Eberhard Wolff
 
Beyond Microservices
Beyond MicroservicesBeyond Microservices
Beyond Microservices
Eberhard Wolff
 
The Frontiers of Continuous Delivery
The Frontiers of Continuous DeliveryThe Frontiers of Continuous Delivery
The Frontiers of Continuous Delivery
Eberhard Wolff
 
Four Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, AsyncFour Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, Async
Eberhard Wolff
 
Microservices - not just with Java
Microservices - not just with JavaMicroservices - not just with Java
Microservices - not just with Java
Eberhard Wolff
 
Deployment - Done Right!
Deployment - Done Right!Deployment - Done Right!
Deployment - Done Right!
Eberhard Wolff
 
Data Architecture not Just for Microservices
Data Architecture not Just for MicroservicesData Architecture not Just for Microservices
Data Architecture not Just for Microservices
Eberhard Wolff
 
How to Split Your System into Microservices
How to Split Your System into MicroservicesHow to Split Your System into Microservices
How to Split Your System into Microservices
Eberhard Wolff
 
Microservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale AgileMicroservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale Agile
Eberhard Wolff
 
How Small Can Java Microservices Be?
How Small Can Java Microservices Be?How Small Can Java Microservices Be?
How Small Can Java Microservices Be?
Eberhard Wolff
 
Data Architecturen Not Just for Microservices
Data Architecturen Not Just for MicroservicesData Architecturen Not Just for Microservices
Data Architecturen Not Just for Microservices
Eberhard Wolff
 
Microservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityMicroservices: Redundancy=Maintainability
Microservices: Redundancy=Maintainability
Eberhard Wolff
 
Self-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesSelf-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to Microservices
Eberhard Wolff
 
Microservices Technology Stack
Microservices Technology StackMicroservices Technology Stack
Microservices Technology Stack
Eberhard Wolff
 
Software Architecture for Innovation
Software Architecture for InnovationSoftware Architecture for Innovation
Software Architecture for Innovation
Eberhard Wolff
 
Five (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous DeliveryFive (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous Delivery
Eberhard Wolff
 
Nanoservices and Microservices with Java
Nanoservices and Microservices with JavaNanoservices and Microservices with Java
Nanoservices and Microservices with Java
Eberhard Wolff
 
Microservices: Architecture to Support Agile
Microservices: Architecture to Support AgileMicroservices: Architecture to Support Agile
Microservices: Architecture to Support Agile
Eberhard Wolff
 
Microservices: Architecture to scale Agile
Microservices: Architecture to scale AgileMicroservices: Architecture to scale Agile
Microservices: Architecture to scale Agile
Eberhard Wolff
 
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three BuzzwordsMicroservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Eberhard Wolff
 

More from Eberhard Wolff (20)

Architectures and Alternatives
Architectures and AlternativesArchitectures and Alternatives
Architectures and Alternatives
 
Beyond Microservices
Beyond MicroservicesBeyond Microservices
Beyond Microservices
 
The Frontiers of Continuous Delivery
The Frontiers of Continuous DeliveryThe Frontiers of Continuous Delivery
The Frontiers of Continuous Delivery
 
Four Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, AsyncFour Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, Async
 
Microservices - not just with Java
Microservices - not just with JavaMicroservices - not just with Java
Microservices - not just with Java
 
Deployment - Done Right!
Deployment - Done Right!Deployment - Done Right!
Deployment - Done Right!
 
Data Architecture not Just for Microservices
Data Architecture not Just for MicroservicesData Architecture not Just for Microservices
Data Architecture not Just for Microservices
 
How to Split Your System into Microservices
How to Split Your System into MicroservicesHow to Split Your System into Microservices
How to Split Your System into Microservices
 
Microservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale AgileMicroservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale Agile
 
How Small Can Java Microservices Be?
How Small Can Java Microservices Be?How Small Can Java Microservices Be?
How Small Can Java Microservices Be?
 
Data Architecturen Not Just for Microservices
Data Architecturen Not Just for MicroservicesData Architecturen Not Just for Microservices
Data Architecturen Not Just for Microservices
 
Microservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityMicroservices: Redundancy=Maintainability
Microservices: Redundancy=Maintainability
 
Self-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesSelf-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to Microservices
 
Microservices Technology Stack
Microservices Technology StackMicroservices Technology Stack
Microservices Technology Stack
 
Software Architecture for Innovation
Software Architecture for InnovationSoftware Architecture for Innovation
Software Architecture for Innovation
 
Five (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous DeliveryFive (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous Delivery
 
Nanoservices and Microservices with Java
Nanoservices and Microservices with JavaNanoservices and Microservices with Java
Nanoservices and Microservices with Java
 
Microservices: Architecture to Support Agile
Microservices: Architecture to Support AgileMicroservices: Architecture to Support Agile
Microservices: Architecture to Support Agile
 
Microservices: Architecture to scale Agile
Microservices: Architecture to scale AgileMicroservices: Architecture to scale Agile
Microservices: Architecture to scale Agile
 
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three BuzzwordsMicroservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
 

Recently uploaded

Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
Fwdays
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 

Recently uploaded (20)

Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 

ELK Stack

  • 1. ELK
  • 2. Eberhard Wolff - @ewolff Docker •  No true virtualization •  Linux Containers (lxc) •  i.e. shared kernel •  i.e. separate file systems
  • 3. Eberhard Wolff - @ewolff Docker File Systems •  Read only base images •  +read/write image •  Can be stacked
  • 4. Eberhard Wolff - @ewolff Docker •  Linux only •  So: Run it in VM •  Controlled by Vagrant
  • 6. Eberhard Wolff - @ewolff Communication Between Docker Container •  Via ports •  Via data volumes
  • 8. Eberhard Wolff - @ewolff Example •  Log file analysis •  ELK Stack •  Elasticsearch: Store •  Logstash: Parse •  Kibana: Web based GUI •  Install three servers •  Automate installation ... L
  • 9. Eberhard Wolff - @ewolff Dockerized ELK Stack •  1 Vagrantfile (25 lines) •  6 Dockerfiles (total 28 lines) •  1 HTML file (11 lines) •  1 Config (Logstash) (29 lines) •  Total: 93 lines, 9 files •  Including one RegExp L
  • 10. Eberhard Wolff - @ewolff Vagrant VM Container user-registration 8080 8081 Port 8081 Container elasticsearch Container logstash 9200 9200 Port 9200 9200 link Container kibana Port 8080 8080 8080 Browser Volume /log
  • 12. Eberhard Wolff - @ewolff date time level 1988-10-12 19:42:07.350 INFO 683 [http-nio-8080-exec-6] --- c.e.u.service.RegistrationService : email=eberhard.wolff@gmail.com geloesc pid thread class logmessage
  • 13. Eberhard Wolff - @ewolff Scaling Elasticsearch •  More servers •  Shards •  Easy to set up •  Index per day or week or month •  Smaller dataset
  • 14. Eberhard Wolff - @ewolff Ideas about Logstash •  No parsing •  Send JSON
  • 15. Eberhard Wolff - @ewolff Break Logstash apart •  Logstash does parsing and communication •  Not too sensible in a distributed sytem •  Separate process per responsibility
  • 16. Eberhard Wolff - @ewolff Shipper •  Ship data from logging system •  Logstash •  Beaver https://github.com/josegonzalez/ beaver •  Woodchuck https://github.com/danryan/ woodchuck
  • 17. Eberhard Wolff - @ewolff Broker •  Buffer data for later processing •  Redis •  In-Memory NoSQL database •  Small •  Fast •  ...and then Parse (Logstash)
  • 18. Eberhard Wolff - @ewolff Links •  https://github.com/ewolff/user- registration
  • 19. Eberhard Wolff - @ewolff Thank You!! eberhard.wolff@gmail.com @ewolff