1. Cloud Application Logging
For Forensics
Raffael Marty - @zrlram
ACM SAC 2011, Taichung - Taiwan
Saturday, March 26, 2011

2. Agenda
Guidelines for logging infrastructures and application development
To enable the forensic process
•The Cloud and its Logs •Logging Architecture
•Logging Guidelines •Reference Setup
Logging as a Service 2 © by Raffael Marty
Saturday, March 26, 2011

3. Logs for Computer Forensics
• Logs are part of the forensic process
• Problems: 2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a
mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/www-
access.log'
2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: Follow-mode file source not found,
- can’t find logs deferring open; filename='/mnt/log/apache2/www-error.log'
2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a
mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/www-
error.log'
- logs got deleted
2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a
mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/
access.log'
2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a
- logs never got generated
mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/
error.log'
- logs are incomplete (e.g., no user name)
- log format is unknown
- archival and retention of logs (how long?)
- knowledge of logging configuration
- non compatible and random log formats (make correlation impossible)
Logging as a Service 3 © by Raffael Marty
Saturday, March 26, 2011

4. The Cloud
IaaS - Infrastructure Complete control OS up
PaaS - Platform No control over OS
SaaS - Software No or very limited control
LaaS - Logging
Logging as a Service 4 © by Raffael Marty
Saturday, March 26, 2011

5. Logs in the Cloud
• Generally no infrastructure logs!
- Routers, Firewalls, Load balancers, etc.
• PaaS only limited access to OS logging
• SaaS generally no access to any logs
• Volatility of machines / logs
• Highly decentralized
Logging as a Service 5 © by Raffael Marty
Saturday, March 26, 2011

6. Applications Enable Visibility
• If you can’t control the infrastructure, control your applications
• Application logging
- needs guidelines
- better tools
- education of developers
and students?
Raffael Marty - @zrlram 6
Saturday, March 26, 2011

7. What?
Mar 16 08:09:58 kernel: [! 0.000000] Normal! 1048576 -> 1048576
Logging as a Service 7 © by Raffael Marty
Saturday, March 26, 2011

8. Logging Guidelines
• When • What • How
8
Saturday, March 26, 2011

9. When to Log
• Operations based logging
- Errors are problems that impact a single application user
- Critical conditions: situations that impacts all users of the application
- System and application start, stop, and restart.
- Changes to objects - attribute changes to an activity
‣ Installation of a new application
‣ Configuration change
‣ Logging program code updates
‣ Backup runs
‣ Audit of log access
Logging as a Service 9 © by Raffael Marty
Saturday, March 26, 2011

10. When to Log
• Security (forensics) related logging
- Login / logout (local and remote)
- Password changes / authorization changes
- Failed resource access (denied authorization)
- All activity executed by a privileged account
• Regulatory and standards mandates
- SOX (Financial system access)
- PCI (Cardholder data access)
- etc.
• Business relevant logging
Logging as a Service 10 © by Raffael Marty
Saturday, March 26, 2011

11. What to Log
when, what, who, and why debug
info
warn
• Timestamp 2010-05-13 13:03:47.123231PDT error
crit
• Severity info
see topic of event
• Categorization object=input, action=create, status=success categorization
• Application loggly-indexing
• User zrlram
across tiers
• Session ID 08BaswoAAQgAADVDG3IAAAAD and applications!
• Reason -
Logging as a Service 11 © by Raffael Marty
Saturday, March 26, 2011

12. How to Log
• Machine processable
- field-identification
- speed
• Field normalization
- ranges (high, medium, low ==> 5, 3, 1)
- terms (dropped, blocked, drop, denied)
• Encoding, see existing standards (e.g., syslog, CEE)
Logging as a Service 12 © by Raffael Marty
Saturday, March 26, 2011

13. Log Formats
- simple text --> key-value
time=2010-05-13 13:03:47.123231PDT,session_id=08AADVDG3IAAAAD,severity=ERROR,user=zrlram,
object=customer,action=delete,status=failure, reason=does not exist
- expressive text --> JSON
{“time”:”2010-05-13 13:03:47.123231PDT”,“session_id”:”08AADVDG3IAAAAD”,
“severity”=”ERROR”,”user”=”zrlram”,“category”: {“object”:”customer”,”action”:”delete”, ”status”:”failure”},
“reason”:”does not exist”}
- binary --> special encoding
Logging as a Service 13 © by Raffael Marty
Saturday, March 26, 2011

14. Logging Architecture
• Enable logging
• Log transport
- tcp vs. udp vs. relp vs. http?
- encryption and compression?
- Synchronized clocks across components
• Centralization of logs
- preserve integrity
• Tune logging configurations
- based on use-cases
Logging as a Service 14 © by Raffael Marty
Saturday, March 26, 2011

15. Reference Setup
Clients: JavaScript
Elastic LoadBalancer
Amazon RDS Frontends: Apache, Django
Backend: Java - log4j
Across machines: collectd, puppet, OS syslog
Logging as a Service 15 © by Raffael Marty
Saturday, March 26, 2011

16. Future Work
Analyzing framework for forensic log analysis
-security visualization (see http://secviz.org)
-forensic timeline analysis
-log review
mobile-166 My syslog
-log correlation
-policy monitoring
Logging as a Service 16 © by Raffael Marty
Saturday, March 26, 2011

17. We are hiring!
about.me/raffy
17
Saturday, March 26, 2011