Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Choosing Your Log Management Approach: Buy, Build or Outsource


Published on

Presentation from Anton Chuvakin on Choosing Your Log Management Approach: Buy, Build or Outsource (was given at SANS many times)

Published in: Technology, Education
  • Be the first to comment

Choosing Your Log Management Approach: Buy, Build or Outsource

  1. 1. How Would You Do It? Selecting a Log Management Approach Anton Chuvakin, Ph.D., GCIH, GCFA Chief Logging Evangelist LogLogic, Inc Mitigating Risk. Automating Compliance.
  2. 2. Outline <ul><li>Are you convinced: why log management? </li></ul><ul><ul><li>Hey, why not just ignore the logs, as usual !  </li></ul></ul><ul><li>Choices, choices: build, buy, outsource, combine… </li></ul><ul><ul><li>Build advantage and risks </li></ul></ul><ul><ul><li>Buy advantage and risks </li></ul></ul><ul><ul><li>Outsource advantage and risks </li></ul></ul><ul><ul><li>Combined strategies </li></ul></ul><ul><li>Critical issues </li></ul><ul><ul><li>Buy: questions to ask your vendor </li></ul></ul><ul><ul><li>Build: open-source tools available </li></ul></ul><ul><li>Conclusions </li></ul>
  3. 3. Log Data Overview <ul><li>Audit logs </li></ul><ul><li>Transaction logs </li></ul><ul><li>Intrusion logs </li></ul><ul><li>Connection logs </li></ul><ul><li>System performance records </li></ul><ul><li>User activity logs </li></ul><ul><li>Various alerts and other messages </li></ul><ul><li>Firewalls/intrusion prevention </li></ul><ul><li>Routers/switches </li></ul><ul><li>Intrusion detection </li></ul><ul><li>Servers, desktops, mainframes </li></ul><ul><li>Business applications </li></ul><ul><li>Databases </li></ul><ul><li>Anti-virus </li></ul><ul><li>VPNs </li></ul>What logs? From Where?
  4. 4. Why Log Management? <ul><li>Threat protection and discovery </li></ul><ul><li>Incident response </li></ul><ul><li>Forensics , “e-discovery” and litigation support </li></ul><ul><li>Regulatory compliance </li></ul><ul><li>Internal policies and procedure compliance </li></ul><ul><li>Internal and external audit support </li></ul><ul><li>IT system and network troubleshooting </li></ul><ul><li>IT performance management </li></ul>
  5. 5. Log Management Mandate and Regulations Regulations Require LMI <ul><li>SOX </li></ul><ul><li>GLBA </li></ul><ul><li>FISMA </li></ul><ul><li>JPA </li></ul><ul><li>NIST 800-53 </li></ul><ul><ul><li>Capture audit records </li></ul></ul><ul><ul><li>Regularly review audit records for unusual activity and violations </li></ul></ul><ul><ul><li>Automatically process audit records </li></ul></ul><ul><ul><li>Protect audit information from unauthorized deletion </li></ul></ul><ul><ul><li>Retain audit logs </li></ul></ul><ul><li>NIST 800-92 Log Management Guide! </li></ul><ul><li>PCI </li></ul><ul><li>HIPAA </li></ul><ul><li>SLAs </li></ul>Mandates Demand It <ul><li>PCI : Requirement 10 and beyond </li></ul><ul><ul><li>Logging and user activities tracking are critical </li></ul></ul><ul><ul><li>Automate and secure audit trails for event reconstruction </li></ul></ul><ul><ul><li>Review logs daily </li></ul></ul><ul><ul><li>Retain audit trail history for at least one year </li></ul></ul><ul><li>COBIT </li></ul><ul><li>ISO </li></ul><ul><li>ITIL </li></ul><ul><li>COBIT 4 </li></ul><ul><ul><li>Provide audit trail for root-cause analysis </li></ul></ul><ul><ul><li>Use logging to detect unusual or abnormal activities </li></ul></ul><ul><ul><li>Regularly review access, privileges, changes </li></ul></ul><ul><ul><li>Verify backup completion </li></ul></ul><ul><li>ISO17799 </li></ul><ul><ul><li>Maintain audit logs for system access and use, changes, faults, corrections, capacity demands </li></ul></ul><ul><ul><li>Review the results of monitoring activities regularly and ensure the accuracy of logs </li></ul></ul>Controls Require it “ Get fined, Get Sanctioned” “ Lose Customers, Reputation, Revenue or Job” “ Get fined, Go To Jail”
  6. 6. Log Management Process
  7. 7. How Do You Do It? <ul><li>Now that you are convinced that log management is A MUST, your choices are: </li></ul><ul><ul><li>Outsource </li></ul></ul><ul><ul><li>Built </li></ul></ul><ul><ul><li>Buy </li></ul></ul><ul><li>Combined strategies are also possible – some offer unique advantages </li></ul>
  8. 8. Outsource <ul><li>Risks </li></ul><ul><li>Somebody else will worry about your problems! </li></ul><ul><li>Requirements not met </li></ul><ul><li>SLA risks and lost control of data </li></ul><ul><li>Volume and log access challenges </li></ul><ul><li>Advantages </li></ul><ul><li>Somebody else will worry about your problems! </li></ul><ul><li>Likely, no need to run any equipment in house </li></ul><ul><li>Less staff needed </li></ul><ul><li>Management will like it  </li></ul>
  9. 9. Outsourcing LM: What to Be Aware Of? <ul><li>Will all your logs be going to the MSSP? What will? </li></ul><ul><ul><li>Likely not – no way to move all! </li></ul></ul><ul><li>Does MSSP have skills to analyze your site-specific logs? </li></ul><ul><ul><li>Probably not … </li></ul></ul><ul><li>Can you still take a peek at your logs? </li></ul><ul><ul><li>Do you need to call for that? </li></ul></ul><ul><ul><li>Can you just review, search, etc your raw logs? </li></ul></ul><ul><li>BTW, SaaS is NOT MSSP – you need to do the work (oh, horror!  ) </li></ul>
  10. 10. Build <ul><li>Risks </li></ul><ul><li>Ongoing maintenance will kill you  </li></ul><ul><li>No support, apart from you </li></ul><ul><li>Does it pass the “ bus test ”? </li></ul><ul><li>Handling log volume </li></ul><ul><li>Will it scale with you? </li></ul><ul><li>Advantages </li></ul><ul><li>Likely will get exactly what you want </li></ul><ul><li>You can do things that no vendor has </li></ul><ul><li>Choose platform, tools, methods </li></ul><ul><li>No up front cost </li></ul><ul><li>Its fun to do!  </li></ul>
  11. 11. Open-Source Pieces That Help! <ul><li>Log collection </li></ul><ul><ul><li>Syslog-ng, kiwi, Snare, Project LASSO, Apache2syslog, logger, etc </li></ul></ul><ul><li>Secure centralization </li></ul><ul><ul><li>Stunnel, ssh/scp, free IPSec VPNs </li></ul></ul><ul><li>Pre-processing </li></ul><ul><ul><li>LogPP – from ugly logs to cute ones  </li></ul></ul><ul><li>Storage </li></ul><ul><ul><li>MySQL or design your own file-based storage </li></ul></ul><ul><li>Analysis – a tough one!  </li></ul><ul><ul><li>MS Excel – yes, still a top choice! </li></ul></ul><ul><ul><li>OSSEC and OSSIM for [ some ] intelligence </li></ul></ul><ul><ul><li>SEC for correlation </li></ul></ul><ul><ul><li>Swatch, logwatch, logsentry, other match-n-bug scripts (too many!) </li></ul></ul>
  12. 12. Buy <ul><li>Risks </li></ul><ul><li>“Cash and carry” – pay and get a tool you need to use now </li></ul><ul><li>Skilled staff needed to get value out of a purchased appliance </li></ul><ul><li>Requirements not met </li></ul><ul><li>Vendor longevity </li></ul><ul><li>Advantages </li></ul><ul><li>“ Cash and carry” – pay and get a “ solution ” </li></ul><ul><li>Support for log sources </li></ul><ul><li>Ongoing improvements, support and guidance </li></ul><ul><li>“ Have a face(s) to scream at!” </li></ul>
  13. 13. Questions to Discuss With Your Vendor <ul><li>Are you collecting and aggregating 100% of all log data from all data sources on the network? </li></ul><ul><li>Are your logs transported and stored securely ? </li></ul><ul><li>Are there packaged reports that suit your needs? Can you create the needed reports to organize collected log data quickly? </li></ul><ul><li>Can you set alerts on anything in the logs? </li></ul><ul><li>Are you looking at log data on a daily basis? Can you prove that you are? </li></ul><ul><li>Can you perform fast, targeted searches for specific data? </li></ul><ul><li>Can you contextualize log data (comparing application, network and database logs) when undertaking forensics and other operational tasks? </li></ul><ul><li>Can you readily prove that security, change management, and access control policies are in use and up to date? </li></ul><ul><li>Can you securely share log data with other applications and users? </li></ul>
  14. 14. Combined Strategies: Often the Best… <ul><li>… but you might need to pay twice  </li></ul><ul><li>Buy + Build : great idea – enhance vendor tools with internal custom development OR combine vendor tools with open-source tools (build, then buy or the opposite ) </li></ul><ul><li>Buy + Outsource : split the work with an MSSP team and retain more control </li></ul><ul><li>Build + Outsource : combine your own with MSSP </li></ul><ul><li>Combined approaches mitigate some of the risks, but at a cost (see TANFL principle  ) </li></ul>
  15. 15. Build + Buy: Surprisingly Effective! <ul><li>Capture buy advantages: </li></ul><ul><ul><li>Support </li></ul></ul><ul><ul><li>Ongoing improvements </li></ul></ul><ul><ul><li>Performance and scalability of the platform </li></ul></ul><ul><ul><li>Routine, boring log management tasks done by vendor! </li></ul></ul><ul><li>Capture build advantages: </li></ul><ul><ul><li>Build analysis you want on top of the vendor platform (e.g. via web API like LogLogic’s) </li></ul></ul><ul><ul><li>Present the data you want to the people that need it </li></ul></ul><ul><ul><li>Fun log management tasks done by you! </li></ul></ul>
  16. 16. Finally, How to Choose? <ul><li>Breadth/depth of project requirements </li></ul><ul><ul><li>Just how unusual you are? </li></ul></ul><ul><ul><li>Unique needs or volumes </li></ul></ul><ul><li>Size of organization </li></ul><ul><li>Available resources </li></ul><ul><ul><li>Money </li></ul></ul><ul><ul><li>Development talent </li></ul></ul><ul><li>Organization culture and management support </li></ul><ul><li>Deployed hardware and software </li></ul><ul><ul><li>Run any Tandem?  </li></ul></ul>
  17. 17. Take Action! <ul><li>Turn ON logging! </li></ul><ul><li>Assess the role of log data in meeting compliance requirements , mitigating security risks , enabling audit and improving availability </li></ul><ul><li>Implement log management strategy as outlined above </li></ul><ul><li>Only “roll your own” after analyzing other options as well as pro/con arguments </li></ul><ul><li>Attend webcasts ( ) and read our blog at </li></ul>
  18. 18. Thank You! <ul><li>Anton Chuvakin, Ph.D., GCIH, GCFA </li></ul><ul><li>Chief Logging Evangelist </li></ul><ul><li>LogLogic, Inc </li></ul><ul><li>See for my papers, books, reviews </li></ul><ul><li>and other security and logging resources. </li></ul><ul><li>Subscribe to my blog at </li></ul>