The document outlines the etiquette expected during a session on Elasticsearch, including punctuality, feedback, and minimizing disturbances. It offers an overview of Elasticsearch, Kibana, and Filebeat, detailing their functionalities and benefits, such as high performance, scalability, and data visualization. Additionally, it highlights the operational aspects of Filebeat as a log data shipper that forwards and centralizes logs for processing.

1. Filebeat-
Elastic Search
Prashant Trivedi

2. Lack of etiquette and manners is a huge turn off.
KnolX Etiquettes
 Punctuality
Join the session 5 minutes prior to the session start time. We start on time
and conclude on time!
 Feedback
Make sure to submit a constructive feedback for all sessions as it is very
helpful for the presenter.
 Silent Mode
Keep your mobile devices in silent mode, feel free to move out of session
in case you need to attend an urgent call.
 Avoid Disturbance
Avoid unwanted chit chat during the session.

3. Agenda
1. Elastic Search
 How it works?
 Benefits
2. Kibana
 Overview
3. Filebeat
 How it works?
 Architectural Fig.
4. Demo

4. Elastic Search

5. Elastic Search
Elasticsearch is a distributed search and analytics engine built
on Apache Lucene. Since its release in 2010, Elasticsearch has
quickly become the most popular search engine and is
commonly used for log analytics, full-text search, security
intelligence, business analytics, and operational intelligence use
cases.

6. How it works?
You can send data in the form of JSON documents to Elasticsearch using
the API or ingestion tools such as Logstash and Amazon kinesis data
firehorse. Elasticsearch automatically stores the original document and
adds a searchable reference to the document in the cluster’s index. You
can then search and retrieve the document using the Elasticsearch API.
You can also use Kibana, a visualization tool, with Elasticsearch to
visualize your data and build interactive dashboards.

7. Benefits
High Performance :-
Elasticsearch provides fast and relevant matches for
full-text searches. Distributed search indices
help retrieve data within a second. It is faster than a
typical SQL database that may take few seconds.
Easily Scalable:-
Elasticsearch is a distributed system by nature. You
can scale to thousands of servers quickly. Add
servers (nodes) to a cluster to increase capacity.
Easy Application Development:-
Application development requires large volumes of
data. Slow database querying and data retrieval are
inefficient. Elasticsearch makes it easy to manage
and store large datasets. It supports various
languages such as Java, Python, PHP, JavaScript,
Node.js, Ruby, and more.
Distributed Approach:-
Elasticsearch works on a distributed architecture. As a result, it
can handle vast amounts of data quickly. The indices are
broken into shards. Shards work as a fully functional index.
Each shard can have many replicas. You can host these
shards anywhere in the Elasticsearch cluster.
Lots of Search Options:-
Elasticsearch offers many features in search. You can get
faceted search, full-text search, auto-complete, instant search,
and more.
Near Real-Time Operations:-
When a document is stored, it is indexed and searched in near
real-time. You get responses to queries in less than one
second. The documents are available immediately after
indexing.

8. Kibana

9. Kibana
 Kibana is a visual interface tool that allows you to explore, visualize, and build a dashboard over the
log data massed in Elasticsearch Clusters. Elastic is the company behind Kibana and the two other
open source tools - Elasticsearch and Logstash. The Elasticsearch tool serves as the database for
document-oriented and semi-structured data.
 The core feature of Kibana is data querying & analysis. In addition, Kibana's visualization features
allow you to visualize data in alternate ways using heat maps, line graphs, histograms, pie charts,
and geospatial support. With various methods, you can search the data stored in Elasticsearch for
root cause diagnostics.
 With Kibana, it is easy to understand big data, and you can quickly build and share dynamic
dashboards that frame-out changes to the Elasticsearch query in real-time. This visualization tool is
equipped with various options

10. Filebeat

11. Filebeat
Filebeat is a lightweight shipper for forwarding and centralizing log data. Installed as an agent on
your servers, Filebeat monitors the log files or locations that you specify, collects log events, and
forwards them either to Elasticsearch or Logstash for indexing.
It has mainly two components:-
 Harvester — A harvester is responsible for reading the content of a single file. The harvester reads each file,
line by line, and sends the content to the output.
 Input — An input is responsible for managing the harvesters and finding all sources to read from

12. How it works?
 It starts with one or more inputs that look in the locations you’ve specified for
log data.
 For each log that Filebeat locates, it starts a harvester.
 Each harvester reads a single log for new content and sends the new log
data to libbeat.
 Libbeat aggregates the events and sends the aggregated data to the output
that you’ve configured for File beat.

14. Demo

15. Thank you