Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

It Shore Beats Working: Configuring Elasticsearch to get the Most out of Cloud Logging

230 views

Published on

Mention the Ipro Centralized login concept and differentiate between that and elastic/logstash

Published in: Law
  • Login to see the comments

  • Be the first to like this

It Shore Beats Working: Configuring Elasticsearch to get the Most out of Cloud Logging

  1. 1. It Shore Beats Working: Configuring Elasticsearch to get the Most out of Cloud Logging
  2. 2. Private and Confidential – Copyright 2019 What’s in it for You? ElasticSearch Centralized logging from a remote endpoint. Easily export relevant log data. Sequential logging from all components. Fast, near real-time searching Easily accessible from any location.
  3. 3. Presentation TitlePrivate and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 What is Elasticsearch? Open source distributed, RESTful search and analytics engine capable of solving a growing number of use cases. • Real-time Full-text Searching capability • Supports Aggregations • Distributed and Scalable functionality • Highly Available and Fault Tolerant • Inverted index structure provides fast document retrieval
  4. 4. Presentation TitlePrivate and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 Centralize Logging with Elasticsearch https://www.elastic.co/downloads/ Log Query
  5. 5. Private and Confidential – Copyright 2019 Elasticsearch Dedicated System Requirements • 64 GB RAM – Recommended • (16-64 GB) 50% Java heap Memory • 2-8 Cores • Concurrency outperforms clock speed CPU • SSD • Use fastest affordable disk configuration possible – Do not use NAS DISK • Gigabit Ethernet (1GbE) Network
  6. 6. Integrating Ipro with Elastic
  7. 7. Private and Confidential – Copyright 2019 Centralize Logging with Elasticsearch Find Installed / Registered components SELECT DISTINCT [Name] ,[HostName] ,[IpAddress1] FROM [ADDConfig].[Enterprise].[InstalledComponent] Grid formatted log information includes: • date/time • machine name • environment • component • type • event status When the log row is clicked, detailed error information appears
  8. 8. Private and Confidential – Copyright 2019 Configuring Elasticsearch Current out of box configuration is NLOG Each application component requires a config file update • C:Program Files (x86)IPRO Tech • C:Program FilesIpro Tech The configuration files are located in the component’s installation directory • C:ProgramDataIPRO Techconfiguration Some components require an additional log configuration update in: ADD System needs to be supplied the Elasticsearch endpoint
  9. 9. Private and Confidential – Copyright 2019 Updating the App.exe.config Files processing Inside the Config Find the  <connectionStrings>  element Locate the attribute <add name=”ElasticSearch” Update the connectionString= property to (http://myelastichost:9200)
  10. 10. Private and Confidential – Copyright 2019 Updating the App.exe.config Files processing Inside the Config Find the  <factoryAdapter>  element Locate the attribute <add key=”target” value =“elastic” /> Uncomment the line to enable logging to elastic
  11. 11. Private and Confidential – Copyright 2019 Updating the App.nlog.config Files processing Inside the config, locate the <rules> element Add a new attribute <logger name=“*” Then add the following properties minlevel=“Trace” writeTo=“elastic” />
  12. 12. Private and Confidential – Copyright 2019 Updating the App.exe.config Files review Inside the Config Find the  <rules>  element Locate the attribute <logger name Set writeTo=“elastic”
  13. 13. Private and Confidential – Copyright 2019 Updating the App.exe.config Files review Inside the Config Find the  <ConnectionStrings>  element Locate the attribute <add name=“ElasticSearch” Update the connectionString= property to (http://myelastichost:9200)
  14. 14. Private and Confidential – Copyright 2019 Choose System from the main menu. In the left pane of the System page, click Settings. Under Configurations in the Elasticsearch Endpoint field, enter the Elasticsearch URL. Click the corresponding Save button. Define Elasticsearch Endpoint
  15. 15. Private and Confidential – Copyright 2019 View Log Information Choose System Select Logs
  16. 16. Private and Confidential – Copyright 2019 View Log Information Cont.
  17. 17. Let’s Have a Looksee
  18. 18. Presentation TitlePrivate and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 You know, for search. • Leveraging Elasticsearch to CENTRALIZE LOGGING for all components • Elasticsearch is the heart of the Elastic Stack and is the ONLY REQUIRED COMPONENT • The components of the Elastic Stack include: Elasticsearch Logstash: Document Enrichment Kibana: Visualization and Search UI
  19. 19. Private and Confidential – Copyright 2019 Logstash Open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite “stash.” Elastic Search! •Derive structure from unstructured data with grok •Decipher geographic coordinates from IP addresses Ease overall processing, independent of the data source, format, or schema.
  20. 20. Presentation TitlePrivate and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 Private and Confidential – Copyright 2019 The Elastic Stack Ipro Log Files Ipro Log Files Ipro Log Files Collect, parse, and transform incoming data Ports: 5044 Logstash output is indexed for searching Ports: 9200 Logstash Elastic Search Visualize and Query Your Data with Kibana (Ports: 5601) View and Search Ipro System Logs
  21. 21. Private and Confidential – Copyright 2019 Streaming Discovery Log Snippet: This is the target line in the log. A Grok filter must be added to the logstash .conf file in order to field this data.
  22. 22. Private and Confidential – Copyright 2019 Configure Logstash with Grok Filters input {stdin{} file{path => "//Machine15/C$/ProgramData/IPRO Tech/logs/Ipro.eCapture.CurrentDiscovery.Service[Premium EDD Driver 1].log"}} filter { grok {match => {"message" => " AProcessing Rate = %{NUMBER:ProcessingRate:float} GB/hour "} }} output { elasticsearch { hosts => ["Machine001:9200"] } stdout {}} input { stdin{} } output { stdout {} } Default Configuration: Logstash.conf Custom Configuration: Logstash.conf https://www.elastic.co/guide/en/logstash/current/plu gins-filters-grok.html#plugins-filters-grok-match http://grokconstructor.appspot.com/do/construction Grok Pattern Resources:
  23. 23. Private and Confidential – Copyright 2019 Kibana

×