Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enterprise Logging and Log Management: Hot Topics by Dr. Anton Chuvakin


Published on

Title: Enterprise Logging and Log Management: Hot Topics Date & Time: Thursday, April 1, 2010, 11:00am Eastern   Capturing log information is critical to IT organizations for many reasons, including for security incident detection and response, and for compliance with numerous regulations and standards. Join one of the foremost experts on log management, Dr. Anton Chuvakin, as we discuss enterprise logging challenges and issues.

Published in: Technology
  • Login to see the comments

Enterprise Logging and Log Management: Hot Topics by Dr. Anton Chuvakin

  1. 1. Enterprise Logging and Log Management: Hot Topics<br />Dr. Anton Chuvakin<br />Security Warrior Consulting<br /><br />April 2010<br />
  2. 2. Note<br />Full recording with voice and Q&A session at the end can be found here.<br />Q&A can also be found on my blog: (search for “open group log hot topics”)<br />
  3. 3. Outline<br />Logs and Logging Intro<br />Log Management Intro<br />Logging Questions – and Answers<br />Log Management Mistakes<br />Conclusions<br />Quick Look at the Future of Logging!<br />
  4. 4. Logging…<br /><ul><li>No standard format
  5. 5. No standard schema, no agreed level of details
  6. 6. No standard meaning
  7. 7. No taxonomy
  8. 8. No standard transport
  9. 9. No shared knowledge on what to log and how
  10. 10. No logging guidance for developers</li></ul>…but you MUST do it!<br />
  11. 11. Log Chaos - Login?<br /><18> Dec 17 15:45:57 ns5xp: NetScreen device_id=ns5xp system-warning-00515: Admin User netscreen has logged on via Telnet from (2002-12-17 15:50:53) <br /><57> Dec 25 00:04:32:%SEC_LOGIN-5-LOGIN_SUCCESS:LoginSuccess [user:anton] [Source:] [localport:23] at 20:55:40 UTC Fri Feb 28 2006<br /><122> Mar 4 09:23:15 localhostsshd[27577]: Accepted password for anton from ::ffff: port 2895 ssh2<br /><13> Fri Mar 17 14:29:38 2006 680 Security SYSTEM User Failure Audit ENTERPRISE Account Logon Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0    Logon account:  POWERUSER   <br />
  12. 12. Log Data Overview<br />From Where?<br />What logs?<br /><ul><li>Firewalls/intrusion prevention
  13. 13. Routers/switches
  14. 14. Intrusion detection
  15. 15. Servers, desktops, mainframes
  16. 16. Business applications
  17. 17. Databases
  18. 18. Anti-virus
  19. 19. VPNs
  20. 20. Audit logs
  21. 21. Transaction logs
  22. 22. Intrusion logs
  23. 23. Connection logs
  24. 24. System performance records
  25. 25. User activity logs
  26. 26. Various alerts and other messages</li></li></ul><li>BTW, Do Your Logs Look Like This?<br />%PIX|ASA-3-713185 Error: Username too long - connection aborted<br />userenv[error] 1030 RCI-CORPwsupx No description available<br />ERROR: transport error 202: send failed: Success<br />Aug 11 09:11:19 xx null pif ? exit! 0 <br />
  27. 27. Cloud to the Rescue?<br />Question: do you think “cloud” will make logging better due to APIs, XML, structured data, etc?<br />Answer: <br /> "If your security and trust models suck now, you'll be pleasantly surprised by the lack of change when you move to cloud“<br />Chris Hoff @ Cisco<br />
  28. 28. Congressional Hearing: Subcommittee on Emerging Threats, Cybersecurity and Science and Technology<br />April 2008<br /><br />“In a free country, you don't have to ask permission for much of anything, but that freedom is buttressed by the certain knowledge that if you sufficiently screw things then up you will have to pay.”<br />Daniel Geer, Sc.D.<br />
  29. 29. Logs = Accountability<br />Accountability<br />Accountability is answerability, enforcement, responsibility, blameworthiness, liability<br />Log Management<br />Log management is collecting, retaining and analyzing audit trails across the organization<br />There is a strong link between accountability and logging<br />B-I-G Picture: Logs as Enabler of Corporate Accountability<br />
  30. 30. Why Log Management?<br />Threat protection and discovery<br />Incident response<br />Forensics, e-discovery and litigation<br />Regulatory compliance<br />Internal policies and procedure compliance<br />Internal and external audit support<br />IT system and network troubleshooting<br />IT performance management<br />
  31. 31. Comp-what?-liance?<br />70-80% of SIEM/log management projects are funded by compliance budgets today<br />PCI DSS tops the charts! (see Requirement 10)<br />“Buy for compliance, use for security + operations” is very common<br />Logging is present in MOST, and are implied by ALL regulations – perfect compliance technology <br />
  32. 32. 11%<br />82%<br /> 8%<br /> 14%<br />77%<br /> 9%<br /> 17%<br />74%<br /> 9%<br /> 15%<br />73%<br /> 12%<br /> 15%<br />69%<br /> 16%<br /> 19%<br />66%<br /> 15%<br /> 17%<br />66%<br /> 17%<br />24%<br />54%<br />22%<br />22%<br />51%<br />28%<br />Use Cases for Log Data Continue to Expand<br />Security detection and remediation<br />Security analysis and forensics<br />Monitoring IT controls for regulatory compliance<br />Troubleshooting IT problems<br />Monitoring end-user behavior<br />Service level/performance management<br />Configuration/change management<br />Monitoring IT administrator behavior<br />Capacity planning<br />Business analysis<br /> 7%<br />90%<br />2%<br />0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%<br /> (Percentage of respondants, N = 123)<br />Yes, we use SIM technologies for this today<br />No, we don’t use SIM technologies for this today, but plan or would like to do so in the future<br />No, we don’t use SIM technologies for this today and have no plans to do so<br />Source: Enterprise Strategy Group, 2007<br />
  33. 33. However…<br />“The company’s server logs recorded only unsuccessful log-in attempts, not successful ones, frustrating a detailed analysis.”<br />
  34. 34. Journey to Log Management<br />What to log – and why? Logging policy<br />Note: BOTH operations + development!<br />Note: sometimes based on ‘what to review?’<br />What to centralize? Log collection <br />What to save? Log retention <br />What to look at? Periodic log review procedures<br />Ad hoc log review happens first<br />What to alert on? Log monitoring<br />
  35. 35. Log Management Maturity Curve<br />
  36. 36. Logging and Log Management Architecture Tips<br />Application Logging Architecture<br />Use cases<br />Internal standards<br />Detailed develop recommendations<br />Use of API/libraries<br />Log ManagementArchitecture<br /><ul><li>Use cases!!! (3 times)
  37. 37. Scope and Phases
  38. 38. Politics and Legal: Unavoidable
  39. 39. Follow maturity curve!</li></li></ul><li>Logging Questions: What to Log?<br />Devices? Systems? Applications?<br />What approach was taken to determine ‘what to log?’<br />What data are you logging and why are you logging it?<br />How you deal with custom log formats, e.g from custom applications?<br />Retention policy: why? How? What? For how long?<br />
  40. 40. Logging Questions: How to Do Log Management and Review?<br />What are your use cases for log management?<br />What motivated you to review logs?<br />What logs are looked at periodically?<br />What logs are looked at only after an incident?<br />What is automated?<br />What tools used for log review? Log management or SIEM?<br />How are they architected?<br />Who reviews logs?<br />
  41. 41. Top Log Management Mistakes<br />Not logging at all.<br />Approaching logs in silo’ed fashion<br />Storing logs for too short a time<br />Prioritizing the log records before collection<br />Ignoring the logs from applications<br />Not looking at the logs<br />Only looking at what youknow is bad<br />Thinking that compliance=log storage<br />
  42. 42. Conclusions<br />Today:<br />The importance of logging will ONLY GROW<br />Start logging – then start collecting logs – then start reviewing and analyzing logs<br />Software architects and developers need to “get” logging; security team will have to guide them<br />Cloud won’t save us: application logging needs to be dealt with, here or in the cloud!<br />Quick Look at the Future:<br />Logging standards are a MUST – and they will happen<br />Pending a global standard - use your own, but standard across your application infrastructure<br />
  43. 43. Questions?<br />Dr. Anton Chuvakin<br /><br />Google Voice: +1-510-771-7106 <br />Site:<br />Blog:<br />LinkedIn:<br />Consulting:<br />Twitter:@anton_chuvakin<br />
  44. 44. More on Anton<br />Book author: “Security Warrior”, “PCI Compliance”, “Information Security Management Handbook”, “Know Your Enemy II”, “Hacker’s Challenge 3”, etc<br />Conference speaker: SANS, FIRST, GFIRST, ISSA, CSI, Interop, many, many others worldwide<br />Standard developer: CEE, CVSS, OVAL, etc<br />Community role: SANS, Honeynet Project, WASC, CSI, ISSA, OSSTMM, InfraGard, ISSA, others<br />Past roles: Researcher, Security Analyst, Strategist, Evangelist, Product Manager, etc<br />Now: Consultant<br />
  45. 45. Security Warrior Consulting Services<br />Logging and log management policy<br />Develop logging policies and processes, log review procedures, workflows and periodic tasks as well as help architect those to solve organization problems <br />Plan and implement log management architecture to support your business cases; develop specific components such as log data collection, filtering, aggregation, retention, log source configuration as well as reporting, review and validation<br />Customize industry “best practices” related to logging and log review to fit your environment, help link these practices to business services and regulations<br />Help integrate SIEM and logging tools and processes into IT and business operations<br />Content development<br />Develop of correlation rules, reports and other content to make your SIEM and log management product more useful more applicable to your risk profile and compliance needs<br />Create and refine policies, procedures and operational practices for logging and log management to satisfy requirements of PCI DSS, HIPAA, NERC, FISMA and other regulations<br />More at<br />