very modular: various inputs, filters and outputs
input: various application log files, but also syslog, stdin, xmpp, log4j
socket, irc, ...
filter: extract semantics (geo info, grok), add information, remove
information, match fields (cidr, dates, numbers, dns, user agent), ...
output: send events to another system such as graphite,
elasticsearch, email, file, stdout, irc, jira, nagios, s3, redis, xmpp, ...
search and analytics engine
stores collected log events in an uniform way
events can be filtered and queried by clients (e.g. kibana)
analytics and search dashboard for elastic
filtering determines what data is used to populate the dashboard,
queries categorise data inside the dashboard
processess technical logging and audit logging
adds information (hostname, environment, application name)
removes information (sensitive details about customers,
transforms information to a more usable form
ship events to redis
large cluster that contains data
one month of history
also hosts kibana files and stores its dashboards
filters based on environment and timestamp (last 24h)
queries for 'error', 'orange cell', specific error codes
rows and panels for optimal screen usage