Cloud Application Logging for Forensics

3,588 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,588
On SlideShare
0
From Embeds
0
Number of Embeds
531
Actions
Shares
0
Downloads
86
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Cloud Application Logging for Forensics

  1. 1. Cloud Application Logging For Forensics Raffael Marty - @zrlram ACM SAC 2011, Taichung - TaiwanSaturday, March 26, 2011
  2. 2. Agenda Guidelines for logging infrastructures and application development To enable the forensic process •The Cloud and its Logs •Logging Architecture •Logging Guidelines •Reference Setup Logging as a Service 2 © by Raffael MartySaturday, March 26, 2011
  3. 3. Logs for Computer Forensics • Logs are part of the forensic process • Problems: 2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename=/mnt/log/apache2/www- access.log 2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: Follow-mode file source not found, - can’t find logs deferring open; filename=/mnt/log/apache2/www-error.log 2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename=/mnt/log/apache2/www- error.log - logs got deleted 2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename=/mnt/log/apache2/ access.log 2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a - logs never got generated mismatching size/inode information, restarting from the beginning; filename=/mnt/log/apache2/ error.log - logs are incomplete (e.g., no user name) - log format is unknown - archival and retention of logs (how long?) - knowledge of logging configuration - non compatible and random log formats (make correlation impossible) Logging as a Service 3 © by Raffael MartySaturday, March 26, 2011
  4. 4. The Cloud IaaS - Infrastructure Complete control OS up PaaS - Platform No control over OS SaaS - Software No or very limited control LaaS - Logging Logging as a Service 4 © by Raffael MartySaturday, March 26, 2011
  5. 5. Logs in the Cloud • Generally no infrastructure logs! - Routers, Firewalls, Load balancers, etc. • PaaS only limited access to OS logging • SaaS generally no access to any logs • Volatility of machines / logs • Highly decentralized Logging as a Service 5 © by Raffael MartySaturday, March 26, 2011
  6. 6. Applications Enable Visibility • If you can’t control the infrastructure, control your applications • Application logging - needs guidelines - better tools - education of developers and students? Raffael Marty - @zrlram 6Saturday, March 26, 2011
  7. 7. What? Mar 16 08:09:58 kernel: [! 0.000000] Normal! 1048576 -> 1048576 Logging as a Service 7 © by Raffael MartySaturday, March 26, 2011
  8. 8. Logging Guidelines • When • What • How 8Saturday, March 26, 2011
  9. 9. When to Log • Operations based logging - Errors are problems that impact a single application user - Critical conditions: situations that impacts all users of the application - System and application start, stop, and restart. - Changes to objects - attribute changes to an activity ‣ Installation of a new application ‣ Configuration change ‣ Logging program code updates ‣ Backup runs ‣ Audit of log access Logging as a Service 9 © by Raffael MartySaturday, March 26, 2011
  10. 10. When to Log • Security (forensics) related logging - Login / logout (local and remote) - Password changes / authorization changes - Failed resource access (denied authorization) - All activity executed by a privileged account • Regulatory and standards mandates - SOX (Financial system access) - PCI (Cardholder data access) - etc. • Business relevant logging Logging as a Service 10 © by Raffael MartySaturday, March 26, 2011
  11. 11. What to Log when, what, who, and why debug info warn • Timestamp 2010-05-13 13:03:47.123231PDT error crit • Severity info see topic of event • Categorization object=input, action=create, status=success categorization • Application loggly-indexing • User zrlram across tiers • Session ID 08BaswoAAQgAADVDG3IAAAAD and applications! • Reason - Logging as a Service 11 © by Raffael MartySaturday, March 26, 2011
  12. 12. How to Log • Machine processable - field-identification - speed • Field normalization - ranges (high, medium, low ==> 5, 3, 1) - terms (dropped, blocked, drop, denied) • Encoding, see existing standards (e.g., syslog, CEE) Logging as a Service 12 © by Raffael MartySaturday, March 26, 2011
  13. 13. Log Formats - simple text --> key-value time=2010-05-13 13:03:47.123231PDT,session_id=08AADVDG3IAAAAD,severity=ERROR,user=zrlram, object=customer,action=delete,status=failure, reason=does not exist - expressive text --> JSON {“time”:”2010-05-13 13:03:47.123231PDT”,“session_id”:”08AADVDG3IAAAAD”, “severity”=”ERROR”,”user”=”zrlram”,“category”: {“object”:”customer”,”action”:”delete”, ”status”:”failure”}, “reason”:”does not exist”} - binary --> special encoding Logging as a Service 13 © by Raffael MartySaturday, March 26, 2011
  14. 14. Logging Architecture • Enable logging • Log transport - tcp vs. udp vs. relp vs. http? - encryption and compression? - Synchronized clocks across components • Centralization of logs - preserve integrity • Tune logging configurations - based on use-cases Logging as a Service 14 © by Raffael MartySaturday, March 26, 2011
  15. 15. Reference Setup Clients: JavaScript Elastic LoadBalancer Amazon RDS Frontends: Apache, Django Backend: Java - log4j Across machines: collectd, puppet, OS syslog Logging as a Service 15 © by Raffael MartySaturday, March 26, 2011
  16. 16. Future Work Analyzing framework for forensic log analysis -security visualization (see http://secviz.org) -forensic timeline analysis -log review mobile-166 My syslog -log correlation -policy monitoring Logging as a Service 16 © by Raffael MartySaturday, March 26, 2011
  17. 17. We are hiring! about.me/raffy 17Saturday, March 26, 2011

×