Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
State of the art logging
1. State of the art logging
Syslog-ng, journal, CEE/Lumberjack and ELSA
Péter Czanik
community manager
Copyright 2013 BalaBit IT Security Ltd.
2. Topics
• No, it is not about cutting trees :-)
• What is syslog? And syslog-ng?
• Free-form messages against name-value pairs
• The new buzzword: journal
• Standardization efforts: CEE/Lumberjack
• Name-value pairs at work: ELSA
Copyright 2013 BalaBit IT Security Ltd.
3. What is syslog?
• Logging: recording events
• Syslog:
- Application: collecting events
- Protocol: forwarding events
Copyright 2013 BalaBit IT Security Ltd.
4. What is syslog-ng?
• “Next Generation” syslog server
• “Swiss army knife” of logging
• More input sources (files, sockets, and so on)
• Better filtering (not only priority, facility)
• Processing (rewrite, normalize, correlate, and so on)
• More destinations (databases, encrypted network, and
so on)
Copyright 2013 BalaBit IT Security Ltd.
5. What is new since 2.0
• 2.0 is best known, but EOL
• Most important new features since 2.0:
- PatternDB and CSV message parsing
- Correlation
- SQL and MongoDB destinations
- JSON formatting
- Modularization
- Multi-threading
• Next: 3.4
- JSON parsing
- More flexible configuration
Copyright 2013 BalaBit IT Security Ltd.
6. Free form log messages
• Most logs are in /var/log
• Most are from syslog (but also wtmp, apache, and so on)
• Most are: date + hostname + text
Mar 11 13:37:56 linux-6965 sshd[4547]: Accepted
keyboard-interactive/pam for root from 127.0.0.1
port 46048 ssh2
• Text = English sentence with some variable parts
• Easy to read
Copyright 2013 BalaBit IT Security Ltd.
7. Why it does not scale?
• Few logs (workstation) → easy to find information
• Many logs (server) → difficult to find information
• Relevant information is presented differently by each
application
• Difficult to process them with scripts
• Answer: structured logging
- Events represented as name value pairs
Copyright 2013 BalaBit IT Security Ltd.
8. Solution from syslog-ng: PatternDB
• Most messages are static texts with some variable parts
embedded
• PatternDB parser:
- Can extract useful information into name-value pairs
- Add status fields based on message text
• Example:
- user=root
- action=login
- status=failure
• It requires patterns
• syslog-ng: name-value pairs inside
Copyright 2013 BalaBit IT Security Ltd.
9. Journal
• The logging component of systemd
• Name-value pairs inside:
- Message
- Trusted properties
- Any additional name-value pairs
• Native support for name-value pair storage
Copyright 2013 BalaBit IT Security Ltd.
10. Journal: the enemy?
• FAQ: Q: is journal the enemy? A: No!
• Journal is limited to Linux/systemd (syslog-ng: all
Linux/BSD/UNIX)
• Journal is local only (syslog-ng: client – server)
• Journal does not filter or process log messages
• Journal + syslog-ng complement each other
• Logs forwarded to syslog-ng through:
/run/systemd/journal/syslog
• syslog-ng can filter, process and forward logs to many
different destinations (one day also to journal)
Copyright 2013 BalaBit IT Security Ltd.
11. CEE
• Journal, syslog-ng, Windows
eventlog, rsyslog, auditd, and so on are based on name-
value pairs
• All use different field names
• Standardization is a must: CEE → Common Event
Expression
• Events: name-value pairs instead of free-form text
- Taxonomy: name-value pairs to describe events (example: status)
- Dictionary: name-value pairs for event parameters (example: user)
• PatternDB can turn free-form messages into CEE
Copyright 2013 BalaBit IT Security Ltd.
12. Lumberjack
• Make CEE happen → implementation
• Coordinated by RedHat
- CEE (Mitre), syslog-ng, rsyslog, and so on
- Open, with high traffic mailing list
- https://fedorahosted.org/lumberjack/
• API(s) to make structured logging easier
• Work on dictionary, taxonomy, transport issues
Copyright 2013 BalaBit IT Security Ltd.
13. Name-value pairs in action: ELSA
• ELSA: Enterprise Log Search and Archive
• Based on syslog-ng, PatternDB and MySQL
• Simple and powerful web GUI
• Extreme scalability
• Patterns focused on network security
(Cisco, Snort, HTTP, Bro, and so on)
Copyright 2013 BalaBit IT Security Ltd.
