Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What I learnt: Elastic search & Kibana : introduction, installtion & configuration


Published on

An introductory tutorial for novice to install and configure Elastic Search and Kibana

Published in: Technology
  • Be the first to comment

What I learnt: Elastic search & Kibana : introduction, installtion & configuration

  1. 1. What I learned …
  2. 2.  ELK stands for › Elastic search › Logstash › Kibana Although a separate project but built to work exceptionally well together by open source vendor elastic. Used by Mozilla, Quora, Wikimedia, foursquare, Github, Netflix, Stack exchange etc
  3. 3.  A java based distributed hence scalable and cross-platform open source search service based on Apache Lucine Search Engine.  Based on HTTP Rest API  Schema less JASON doc  Real time data insight
  4. 4.  Data Normalization tool (Collect, Enrich & Transport Data)  Fast track option for time consuming ETL  Normalize any type of log (system log, web server log, error log, app log etc) irrespective of data source like Apache or IIS.
  5. 5.  data visualization platform  stunning, powerful graphics from histograms to geomaps.
  6. 6.  Node › Node is s single machine/server stores searchable data › Participate in cluster indexing and search capabilities Node 1 Node 2 Node 3 Node4 Node 5 CLUSTER 1
  7. 7.  Cluster › Collection of nodes › Contains one or more than one nodes › Node contains all data  Index › Collection of documents › E.g. product, account, movies › Identified by name (lowercase)  Used in indexing, updating, deleting, searching docs within index › Can have as many as index within cluster
  8. 8.  Type  A convenient way to store several types of data in same index.  multiple types may live in the same index as long as their fields do not conflict  It is stored within metadata with fixed name  _type  Mapping › Like scheme for table in RDBMS › Describes fields › Include information on how fields should be indexed * Bit confused ! Do not worry see the next slide , hopefully concept will be clear
  9. 9.  Document is basic unit of info that can be indexed , expressed in jason format.  Can be many documents in an index
  10. 10.  Shard › When an index contains large amount of data e. 500 GB or 1 TB, then it is divided into multiple pieces called Shard › Fully functional and independent index that can be stored on any node in a cluster.  Replica › Copy of a shard › Take over if shard fails › By default elastic search adds 5 primary and 1 replica shards.
  11. 11.  only requirement for installing Elastic search is a recent version of Java  Install java › Make sure to configure the run-time environment › Set JAVA_HOME under Advance Settings->environment variable *if it is already added , add the path of java installation folder, if not add new variable. See the next slide
  12. 12.  Download elastic seach for folloing links ›  Download the zip file, Unzip it into C: for convenient  Open command prompt , go to unzipped folder and run binelasticsearch.bat  * you might get JAVA_HOME path problem as I faced
  13. 13.  Now go to browser and type  http://localhost:9200  Check the output is , if the output is like below.. CONGRATS.. Its working fine.
  14. 14.  Download Kibana (.zip) from below link ›  Unzip in to C drive  Now you need to do some configuration  Go to C:kibana-4.5.2-windowsconfig  You will find kibana.yml  Delete the ‘#’ before ‘elasticsearch.url: "http://localhost:9200" ‘
  15. 15.  Now run  Now go to browser and type › http://localhost:5601. if you see the below screen. Cool! Kibana is working perfectly.