Implementing a Log monitoring
Why it is important
• Enterprise systems produce large , different log files.
• When you have lot of logs it is very difficult to trace an issue
or incident .
• Using log files it is possible to alert users when a critical
issue is occurred.
Features of a good log monitoring
• Providing a centralized server , where log from
nodes are possible to upload
• Ability to search specific terms in logs.
• Produce alerts on specific log event, ex : when log
priority is “ERROR”.
• Add a SocketAppender named LOGSTASH to the applications lo4j.xml
<appender name="LOGSTASH" class="org.apache.log4j.net.SocketAppender">
<param name="RemoteHost" value="your_logstash_host_address" />
<param name="ReconnectionDelay" value="60000" />
<param name="Threshold" value="DEBUG" />
• Logstash will collect logs , parse them , and store
them for later use (like , searching).
type => "log4j-type"
port => 4560
host => localhost
port => 9300
cluster => elasticsearch
• Elastic search is used to store and index logs.
• Elastic search
Build on top Lucene
Store as structured JSON documents
Real time data and analytics
• Kibana will be use to search the logs
• Using Kibana, user’s can
– Search the logs
– Visually analyze trends in log vloume to find peaks and
– Score, trend and average fields to find patterns
– Providning RS feed and get updates at any interval