SlideShare a Scribd company logo

ELK Stack

•
•
Eberhard Wolff
Eberhard Wolff

Short presentation about the ELK stack (Elasticsearch, Logstash, Kibana) running on top of Docker / Vagrant.

Technology
1 of 19
Download to read offline
ELK
Eberhard Wolff - @ewolff Docker •  No true virtualization •  Linux Containers (lxc) •  i.e. shared kernel •  i.e. separate file systems
Eberhard Wolff - @ewolff Docker File Systems •  Read only base images •  +read/write image •  Can be stacked
Eberhard Wolff - @ewolff Docker •  Linux only •  So: Run it in VM •  Controlled by Vagrant
Docker Demo
Eberhard Wolff - @ewolff Communication Between Docker Container •  Via ports •  Via data volumes
Docker is a Component System
Eberhard Wolff - @ewolff Example •  Log file analysis •  ELK Stack •  Elasticsearch: Store •  Logstash: Parse •  Kibana: Web based GUI •  Install three servers •  Automate installation ... L
Eberhard Wolff - @ewolff Dockerized ELK Stack •  1 Vagrantfile (25 lines) •  6 Dockerfiles (total 28 lines) •  1 HTML file (11 lines) •  1 Config (Logstash) (29 lines) •  Total: 93 lines, 9 files •  Including one RegExp L
Eberhard Wolff - @ewolff Vagrant VM Container user-registration 8080 8081 Port 8081 Container elasticsearch Container logstash 9200 9200 Port 9200 9200 link Container kibana Port 8080 8080 8080 Browser Volume /log
Demo ELK Stack
Eberhard Wolff - @ewolff date time level 1988-10-12 19:42:07.350 INFO 683 [http-nio-8080-exec-6] --- c.e.u.service.RegistrationService : email=eberhard.wolff@gmail.com geloesc pid thread class logmessage
Eberhard Wolff - @ewolff Scaling Elasticsearch •  More servers •  Shards •  Easy to set up •  Index per day or week or month •  Smaller dataset
Eberhard Wolff - @ewolff Ideas about Logstash •  No parsing •  Send JSON
Eberhard Wolff - @ewolff Break Logstash apart •  Logstash does parsing and communication •  Not too sensible in a distributed sytem •  Separate process per responsibility
Eberhard Wolff - @ewolff Shipper •  Ship data from logging system •  Logstash •  Beaver https://github.com/josegonzalez/ beaver •  Woodchuck https://github.com/danryan/ woodchuck
Eberhard Wolff - @ewolff Broker •  Buffer data for later processing •  Redis •  In-Memory NoSQL database •  Small •  Fast •  ...and then Parse (Logstash)
Eberhard Wolff - @ewolff Links •  https://github.com/ewolff/user- registration
Eberhard Wolff - @ewolff Thank You!! eberhard.wolff@gmail.com @ewolff

Recommended

Elk
Elk Elk
Elk Caleb Wang
 
ELK Stack
ELK StackELK Stack
ELK StackPhuc Nguyen
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack IntroductionVikram Shinde
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elkRushika Shah
 
ELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log ManagementELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log ManagementEl Mahdi Benzekri
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to KibanaVineet .
 
Introducing ELK
Introducing ELKIntroducing ELK
Introducing ELKAllBits BVBA (freelancer)
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackRohit Sharma
 

Recommended

Elk
Elk Elk
Elk Caleb Wang
 
ELK Stack
ELK StackELK Stack
ELK StackPhuc Nguyen
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack IntroductionVikram Shinde
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elkRushika Shah
 
ELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log ManagementELK Elasticsearch Logstash and Kibana Stack for Log Management
ELK Elasticsearch Logstash and Kibana Stack for Log ManagementEl Mahdi Benzekri
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to KibanaVineet .
 
Introducing ELK
Introducing ELKIntroducing ELK
Introducing ELKAllBits BVBA (freelancer)
 
Centralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackCentralized Logging System Using ELK Stack
Centralized Logging System Using ELK StackRohit Sharma
 
Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack PresentationAmr Alaa Yassen
 
The Elastic ELK Stack
The Elastic ELK StackThe Elastic ELK Stack
The Elastic ELK Stackenterprisesearchmeetup
 
Logstash
LogstashLogstash
Logstash琛琳 饶
 
Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaSpringPeople
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELKGeert Pante
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackRich Lee
 
Kibana overview
Kibana overviewKibana overview
Kibana overviewRinat Tainov
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stackVikrant Chauhan
 
Elk - An introduction
Elk - An introductionElk - An introduction
Elk - An introductionHossein Shemshadi
 
elk_stack_alexander_szalonnas
elk_stack_alexander_szalonnaselk_stack_alexander_szalonnas
elk_stack_alexander_szalonnasAlexander Szalonnas
 
Elk stack
Elk stackElk stack
Elk stackJilles van Gurp
 
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...Flink Forward
 
Elasticsearch for beginners
Elasticsearch for beginnersElasticsearch for beginners
Elasticsearch for beginnersNeil Baker
 
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashKeeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashAmazon Web Services
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Edureka!
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...Edureka!
 
Elasticsearch
ElasticsearchElasticsearch
ElasticsearchDivij Sehgal
 
ELK introduction
ELK introductionELK introduction
ELK introductionWaldemar Neto
 
Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips confluent
 
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Edureka!
 
Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is BeautifulEberhard Wolff
 
Java Architectures - a New Hope
Java Architectures - a New HopeJava Architectures - a New Hope
Java Architectures - a New HopeEberhard Wolff
 

More Related Content

What's hot

Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack PresentationAmr Alaa Yassen
 
Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaSpringPeople
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELKGeert Pante
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackRich Lee
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stackVikrant Chauhan
 
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...Flink Forward
 
Elasticsearch for beginners
Elasticsearch for beginnersElasticsearch for beginners
Elasticsearch for beginnersNeil Baker
 
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashKeeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashAmazon Web Services
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Edureka!
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...Edureka!
 
Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips confluent
 
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Edureka!
 

What's hot (20)

Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
 
The Elastic ELK Stack
The Elastic ELK StackThe Elastic ELK Stack
The Elastic ELK Stack
 
Logstash
LogstashLogstash
Logstash
 
Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & Kibana
 
Log management with ELK
Log management with ELKLog management with ELK
Log management with ELK
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
 
Kibana overview
Kibana overviewKibana overview
Kibana overview
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
 
Elk - An introduction
Elk - An introductionElk - An introduction
Elk - An introduction
 
elk_stack_alexander_szalonnas
elk_stack_alexander_szalonnaselk_stack_alexander_szalonnas
elk_stack_alexander_szalonnas
 
Elk stack
Elk stackElk stack
Elk stack
 
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...
 
Elasticsearch for beginners
Elasticsearch for beginnersElasticsearch for beginners
Elasticsearch for beginners
 
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashKeeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
 
Elasticsearch
ElasticsearchElasticsearch
Elasticsearch
 
ELK introduction
ELK introductionELK introduction
ELK introduction
 
Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips
 
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
 

Similar to ELK Stack

Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is BeautifulEberhard Wolff
 
Java Architectures - a New Hope
Java Architectures - a New HopeJava Architectures - a New Hope
Java Architectures - a New HopeEberhard Wolff
 
NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?Eberhard Wolff
 
Micro Service – The New Architecture Paradigm
Micro Service – The New Architecture ParadigmMicro Service – The New Architecture Paradigm
Micro Service – The New Architecture ParadigmEberhard Wolff
 
High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...Eberhard Wolff
 
Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?Eberhard Wolff
 
DockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and DockerDockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and DockerTaylor Brown
 
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...Docker, Inc.
 
Java Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short VersionJava Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short VersionEberhard Wolff
 
Java Application Servers Are Dead!
Java Application Servers Are Dead!Java Application Servers Are Dead!
Java Application Servers Are Dead!Eberhard Wolff
 
Microservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring CloudMicroservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring CloudEberhard Wolff
 
Openstack Cactus Survey
Openstack Cactus SurveyOpenstack Cactus Survey
Openstack Cactus SurveyPjack Chen
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to DockerAlan Forbes
 
Spring Boot
Spring BootSpring Boot
Spring Bootgedoplan
 
Docker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker NetworkingDocker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker NetworkingDocker, Inc.
 
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anyninesCloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anyninesanynines GmbH
 
Magnum Networking Update
Magnum Networking UpdateMagnum Networking Update
Magnum Networking UpdateDaneyon Hansen
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with DockerRavindu Fernando
 
Amazon Elastic Beanstalk
Amazon Elastic BeanstalkAmazon Elastic Beanstalk
Amazon Elastic BeanstalkEberhard Wolff
 

Similar to ELK Stack (20)

Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is Beautiful
 
Java Architectures - a New Hope
Java Architectures - a New HopeJava Architectures - a New Hope
Java Architectures - a New Hope
 
NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?NoSQL Riak MongoDB Elasticsearch - All The Same?
NoSQL Riak MongoDB Elasticsearch - All The Same?
 
Micro Service – The New Architecture Paradigm
Micro Service – The New Architecture ParadigmMicro Service – The New Architecture Paradigm
Micro Service – The New Architecture Paradigm
 
High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...High Availability and Scalability: Too Expensive! Architectures for Future E...
High Availability and Scalability: Too Expensive! Architectures for Future E...
 
Docker Dojo
Docker DojoDocker Dojo
Docker Dojo
 
Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?Microservice - All is Small, All is Well?
Microservice - All is Small, All is Well?
 
DockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and DockerDockerCon 2016 - Windows Server and Docker
DockerCon 2016 - Windows Server and Docker
 
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
Windows Server and Docker - The Internals Behind Bringing Docker and Containe...
 
Java Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short VersionJava Application Servers Are Dead! - Short Version
Java Application Servers Are Dead! - Short Version
 
Java Application Servers Are Dead!
Java Application Servers Are Dead!Java Application Servers Are Dead!
Java Application Servers Are Dead!
 
Microservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring CloudMicroservice With Spring Boot and Spring Cloud
Microservice With Spring Boot and Spring Cloud
 
Openstack Cactus Survey
Openstack Cactus SurveyOpenstack Cactus Survey
Openstack Cactus Survey
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Spring Boot
Spring BootSpring Boot
Spring Boot
 
Docker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker NetworkingDocker Online Meetup #22: Docker Networking
Docker Online Meetup #22: Docker Networking
 
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anyninesCloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
Cloud Infrastructures Slide Set 7 - Docker - Neo4j | anynines
 
Magnum Networking Update
Magnum Networking UpdateMagnum Networking Update
Magnum Networking Update
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with Docker
 
Amazon Elastic Beanstalk
Amazon Elastic BeanstalkAmazon Elastic Beanstalk
Amazon Elastic Beanstalk
 

More from Eberhard Wolff

Architectures and Alternatives
Architectures and AlternativesArchitectures and Alternatives
Architectures and AlternativesEberhard Wolff
 
The Frontiers of Continuous Delivery
The Frontiers of Continuous DeliveryThe Frontiers of Continuous Delivery
The Frontiers of Continuous DeliveryEberhard Wolff
 
Four Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, AsyncFour Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, AsyncEberhard Wolff
 
Microservices - not just with Java
Microservices - not just with JavaMicroservices - not just with Java
Microservices - not just with JavaEberhard Wolff
 
Deployment - Done Right!
Deployment - Done Right!Deployment - Done Right!
Deployment - Done Right!Eberhard Wolff
 
Data Architecture not Just for Microservices
Data Architecture not Just for MicroservicesData Architecture not Just for Microservices
Data Architecture not Just for MicroservicesEberhard Wolff
 
How to Split Your System into Microservices
How to Split Your System into MicroservicesHow to Split Your System into Microservices
How to Split Your System into MicroservicesEberhard Wolff
 
Microservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale AgileMicroservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale AgileEberhard Wolff
 
How Small Can Java Microservices Be?
How Small Can Java Microservices Be?How Small Can Java Microservices Be?
How Small Can Java Microservices Be?Eberhard Wolff
 
Data Architecturen Not Just for Microservices
Data Architecturen Not Just for MicroservicesData Architecturen Not Just for Microservices
Data Architecturen Not Just for MicroservicesEberhard Wolff
 
Microservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityMicroservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityEberhard Wolff
 
Self-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesSelf-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesEberhard Wolff
 
Microservices Technology Stack
Microservices Technology StackMicroservices Technology Stack
Microservices Technology StackEberhard Wolff
 
Software Architecture for Innovation
Software Architecture for InnovationSoftware Architecture for Innovation
Software Architecture for InnovationEberhard Wolff
 
Five (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous DeliveryFive (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous DeliveryEberhard Wolff
 
Nanoservices and Microservices with Java
Nanoservices and Microservices with JavaNanoservices and Microservices with Java
Nanoservices and Microservices with JavaEberhard Wolff
 
Microservices: Architecture to Support Agile
Microservices: Architecture to Support AgileMicroservices: Architecture to Support Agile
Microservices: Architecture to Support AgileEberhard Wolff
 
Microservices: Architecture to scale Agile
Microservices: Architecture to scale AgileMicroservices: Architecture to scale Agile
Microservices: Architecture to scale AgileEberhard Wolff
 
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three BuzzwordsMicroservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three BuzzwordsEberhard Wolff
 

More from Eberhard Wolff (20)

Architectures and Alternatives
Architectures and AlternativesArchitectures and Alternatives
Architectures and Alternatives
 
Beyond Microservices
Beyond MicroservicesBeyond Microservices
Beyond Microservices
 
The Frontiers of Continuous Delivery
The Frontiers of Continuous DeliveryThe Frontiers of Continuous Delivery
The Frontiers of Continuous Delivery
 
Four Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, AsyncFour Times Microservices - REST, Kubernetes, UI Integration, Async
Four Times Microservices - REST, Kubernetes, UI Integration, Async
 
Microservices - not just with Java
Microservices - not just with JavaMicroservices - not just with Java
Microservices - not just with Java
 
Deployment - Done Right!
Deployment - Done Right!Deployment - Done Right!
Deployment - Done Right!
 
Data Architecture not Just for Microservices
Data Architecture not Just for MicroservicesData Architecture not Just for Microservices
Data Architecture not Just for Microservices
 
How to Split Your System into Microservices
How to Split Your System into MicroservicesHow to Split Your System into Microservices
How to Split Your System into Microservices
 
Microservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale AgileMicroservices and Self-contained System to Scale Agile
Microservices and Self-contained System to Scale Agile
 
How Small Can Java Microservices Be?
How Small Can Java Microservices Be?How Small Can Java Microservices Be?
How Small Can Java Microservices Be?
 
Data Architecturen Not Just for Microservices
Data Architecturen Not Just for MicroservicesData Architecturen Not Just for Microservices
Data Architecturen Not Just for Microservices
 
Microservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityMicroservices: Redundancy=Maintainability
Microservices: Redundancy=Maintainability
 
Self-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesSelf-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to Microservices
 
Microservices Technology Stack
Microservices Technology StackMicroservices Technology Stack
Microservices Technology Stack
 
Software Architecture for Innovation
Software Architecture for InnovationSoftware Architecture for Innovation
Software Architecture for Innovation
 
Five (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous DeliveryFive (easy?) Steps Towards Continuous Delivery
Five (easy?) Steps Towards Continuous Delivery
 
Nanoservices and Microservices with Java
Nanoservices and Microservices with JavaNanoservices and Microservices with Java
Nanoservices and Microservices with Java
 
Microservices: Architecture to Support Agile
Microservices: Architecture to Support AgileMicroservices: Architecture to Support Agile
Microservices: Architecture to Support Agile
 
Microservices: Architecture to scale Agile
Microservices: Architecture to scale AgileMicroservices: Architecture to scale Agile
Microservices: Architecture to scale Agile
 
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three BuzzwordsMicroservices, DevOps, Continuous Delivery – More Than Three Buzzwords
Microservices, DevOps, Continuous Delivery – More Than Three Buzzwords
 

Recently uploaded

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 

Recently uploaded (20)

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 

ELK Stack

  • 1. ELK
  • 2. Eberhard Wolff - @ewolff Docker •  No true virtualization •  Linux Containers (lxc) •  i.e. shared kernel •  i.e. separate file systems
  • 3. Eberhard Wolff - @ewolff Docker File Systems •  Read only base images •  +read/write image •  Can be stacked
  • 4. Eberhard Wolff - @ewolff Docker •  Linux only •  So: Run it in VM •  Controlled by Vagrant
  • 6. Eberhard Wolff - @ewolff Communication Between Docker Container •  Via ports •  Via data volumes
  • 8. Eberhard Wolff - @ewolff Example •  Log file analysis •  ELK Stack •  Elasticsearch: Store •  Logstash: Parse •  Kibana: Web based GUI •  Install three servers •  Automate installation ... L
  • 9. Eberhard Wolff - @ewolff Dockerized ELK Stack •  1 Vagrantfile (25 lines) •  6 Dockerfiles (total 28 lines) •  1 HTML file (11 lines) •  1 Config (Logstash) (29 lines) •  Total: 93 lines, 9 files •  Including one RegExp L
  • 10. Eberhard Wolff - @ewolff Vagrant VM Container user-registration 8080 8081 Port 8081 Container elasticsearch Container logstash 9200 9200 Port 9200 9200 link Container kibana Port 8080 8080 8080 Browser Volume /log
  • 12. Eberhard Wolff - @ewolff date time level 1988-10-12 19:42:07.350 INFO 683 [http-nio-8080-exec-6] --- c.e.u.service.RegistrationService : email=eberhard.wolff@gmail.com geloesc pid thread class logmessage
  • 13. Eberhard Wolff - @ewolff Scaling Elasticsearch •  More servers •  Shards •  Easy to set up •  Index per day or week or month •  Smaller dataset
  • 14. Eberhard Wolff - @ewolff Ideas about Logstash •  No parsing •  Send JSON
  • 15. Eberhard Wolff - @ewolff Break Logstash apart •  Logstash does parsing and communication •  Not too sensible in a distributed sytem •  Separate process per responsibility
  • 16. Eberhard Wolff - @ewolff Shipper •  Ship data from logging system •  Logstash •  Beaver https://github.com/josegonzalez/ beaver •  Woodchuck https://github.com/danryan/ woodchuck
  • 17. Eberhard Wolff - @ewolff Broker •  Buffer data for later processing •  Redis •  In-Memory NoSQL database •  Small •  Fast •  ...and then Parse (Logstash)
  • 18. Eberhard Wolff - @ewolff Links •  https://github.com/ewolff/user- registration
  • 19. Eberhard Wolff - @ewolff Thank You!! eberhard.wolff@gmail.com @ewolff