What Every Organization Should Log And Monitor


Published on

My Old MISTI Presentation called "What Every Organization Should Log And Monitor", old but still very useful

Published in: Technology
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Note the switch; you log first and monitor second! I am not an auditor – value the security prospective.
  • What Every Organization Should Log And Monitor

    1. 1. What Every Organization Should Log and Monitor: A Checklist? <ul><li>Anton Chuvakin, Ph.D., GCIA, GCIH </li></ul><ul><li>Security Strategist </li></ul><ul><li>November 15, 2004 </li></ul>
    2. 2. WARNING! <ul><li>This presentation is from 2004. Now in 2008, I might not share all the view expressed in the presentation. </li></ul><ul><li>It is posted the way it was originally presented in the hopes of being useful for somebody. </li></ul>
    3. 3. Highlights <ul><li>Monitoring and logging overview </li></ul><ul><li>Log consolidation strategy: which log sources to include first </li></ul><ul><li>Monitoring and event response strategy </li></ul><ul><li>Log correlation to automate the monitoring </li></ul><ul><li>Using logs for forensics and incident response </li></ul><ul><li>Management and compliance reporting </li></ul>
    4. 4. Definitions <ul><li>Logging </li></ul><ul><li>Auditing </li></ul><ul><li>Monitoring </li></ul><ul><li>Event reporting </li></ul><ul><li>Log analysis </li></ul><ul><li>Alerting </li></ul>
    5. 5. Security Data Overview <ul><li>Audit logs </li></ul><ul><li>Transaction logs </li></ul><ul><li>Intrusion logs </li></ul><ul><li>Connection logs </li></ul><ul><li>System performance records </li></ul><ul><li>User activity logs </li></ul><ul><li>Various alerts </li></ul><ul><li>Firewalls/intrusion prevention </li></ul><ul><li>Routers/switches </li></ul><ul><li>Intrusion detection </li></ul><ul><li>Hosts </li></ul><ul><li>Business applications </li></ul><ul><li>Anti-virus </li></ul><ul><li>VPNs </li></ul>What data? From Where?
    6. 6. Value of Logging and Monitoring <ul><li>Monitoring </li></ul><ul><li>Incident detection </li></ul><ul><li>Loss prevention </li></ul><ul><li>Compliance </li></ul><ul><li>Logging </li></ul><ul><li>Audit </li></ul><ul><li>Forensics </li></ul><ul><li>Incident response </li></ul><ul><li>Compliance </li></ul><ul><li>Analysis </li></ul><ul><li>Deeper insight </li></ul><ul><li>Internal attacks </li></ul><ul><li>Fault prediction </li></ul>
    7. 7. Log Management Process <ul><li>Collect the data </li></ul><ul><li>Convert to a common format </li></ul><ul><li>Reduce in size, if possible </li></ul><ul><li>Transport securely to a central location </li></ul><ul><li>Process in real-time </li></ul><ul><li>Eliminate false positives </li></ul><ul><li>Alert on threats </li></ul><ul><li>Store securely </li></ul><ul><li>Report on trends </li></ul>
    8. 8. Log Process Overview
    9. 9. Centralize the Logs! <ul><li>Accessibility </li></ul><ul><ul><li>All audit records in one place </li></ul></ul><ul><li>Cross-device searchability and analysis </li></ul><ul><ul><li>Categorization </li></ul></ul><ul><ul><li>Correlation </li></ul></ul><ul><li>De-duplication / volume reduction </li></ul><ul><li>Reduced response time </li></ul><ul><li>Increase in the efficiency of existing security point solutions </li></ul>
    10. 10. Retention Time Question <ul><li>I have the answer!  No, not really. </li></ul><ul><li>Regulations? </li></ul><ul><ul><li>Unambiguous: PCI – keep’em for 1 year </li></ul></ul><ul><li>Tiered retention strategy </li></ul><ul><ul><li>Online </li></ul></ul><ul><ul><li>Nearline </li></ul></ul><ul><ul><li>Offline/tape </li></ul></ul>
    11. 11. Monitoring or Ignoring Logs? <ul><li>How to plan a response strategy to activate when monitoring? </li></ul><ul><li>Where to start? </li></ul><ul><li>How to tune it? </li></ul>
    12. 12. Monitoring Strategy
    13. 13. Setting Up Log Monitoring Program <ul><li>Phased approach </li></ul><ul><li>Security gear to connect </li></ul><ul><ul><li>E.g.: DMZ, then core, then other internal systems </li></ul></ul><ul><li>Log types to integrate </li></ul><ul><ul><li>E.g.: IDS (with vulnerability data), then firewalls, then hosts, then others </li></ul></ul><ul><li>Log management components to deploy </li></ul><ul><ul><li>E.g.: collection, reporting, correlation, incident management, others </li></ul></ul><ul><li>Growth of user community </li></ul><ul><ul><li>E.g.: security team, then IT or auditors </li></ul></ul>
    14. 14. Challenges to Deployment <ul><li>Organization political boundaries </li></ul><ul><ul><li>Inherent in any project involving “integration” </li></ul></ul><ul><li>Data crossing network and state boundaries </li></ul><ul><ul><li>Potentially subject to data privacy law </li></ul></ul><ul><li>Access to remote locations where the data sources are </li></ul><ul><ul><li>Remote management, but not remote installation </li></ul></ul><ul><li>Custom applications </li></ul><ul><ul><li>Unsupported and undocumented log formats </li></ul></ul><ul><li>Defined and current escalation trees for incidents </li></ul><ul><ul><li>Who would act on the alert? How is change management handled? </li></ul></ul>
    15. 15. Timing is everything! <ul><li>Timing requirements for analysis </li></ul><ul><li>Real-time fallacy: “we have to have it when?”  </li></ul><ul><li>Log review vs alert monitoring: different challenges and different timing </li></ul>
    16. 16. “Real-Time” Tasks <ul><li>Malware outbreaks </li></ul><ul><li>Convincing and reliable intrusion evidence </li></ul><ul><li>Serious internal network abuse </li></ul><ul><li>Loss of service on critical assets </li></ul>
    17. 17. Daily Tasks <ul><li>Unauthorized configuration changes </li></ul><ul><li>Disruption in other services </li></ul><ul><li>Intrusion evidence </li></ul><ul><li>Suspicious login failures </li></ul><ul><li>Minor malware activity </li></ul><ul><li>Activity summary </li></ul>
    18. 18. Weekly Tasks <ul><li>Review inside and perimeter log trends and activities </li></ul><ul><li>Account creation/removal </li></ul><ul><li>Other host and network device changes </li></ul><ul><li>Less critical attack and probe summary </li></ul>
    19. 19. Monthly Tasks <ul><li>Review long-term network and perimeter trends </li></ul><ul><li>Minor policy violation summary </li></ul><ul><li>Incident team performance measurements </li></ul><ul><li>Security technology performance measurements </li></ul>
    20. 20. “On Incident” Tasks <ul><li>Use SANS six-step incident workflow </li></ul><ul><li>Review all relevant logs on a central logging system </li></ul><ul><li>Collect additional logs, if needed </li></ul>
    21. 21. Reporting <ul><li>Operations </li></ul><ul><ul><li>Reports for Level 1 personnel </li></ul></ul><ul><li>Analytic </li></ul><ul><ul><li>Deep analysis reports </li></ul></ul><ul><li>Management </li></ul><ul><ul><li>“ Boss pleasers”  </li></ul></ul>
    22. 22. Logs in Support of Compliance <ul><li>Application and asset risk measurement </li></ul><ul><li>Data collection and storage to satisfy auditing of controls requirements </li></ul><ul><li>Support for security metrics </li></ul><ul><li>Documented incident resolution procedures </li></ul><ul><li>Industry best-practices for incident management and reporting </li></ul><ul><li>Proof of security due diligence </li></ul><ul><li>Example regulation include: HI PAA , SOX, GLBA,… </li></ul>
    23. 23. Logs for Forensics <ul><li>What? You think this is evidence? Bua-ha-ha-ha  </li></ul><ul><li>“ Computer Records and the Federal Rules of Evidence “ </li></ul><ul><li>“ First , parties may challenge the authenticity of both computer-generated and computer-stored records by questioning whether the records were altered, manipulated, or damaged after they were created. </li></ul><ul><li>Second , parties may question the authenticity of computer-generated records by challenging the reliability of the computer program that generated the records. </li></ul><ul><li>Third , parties may challenge the authenticity of computer-stored records by questioning the identity of their author.” </li></ul>
    24. 24. Logging Device Highlights Usage metrics, violations Application Clean status, update failures Anti-virus Failures, crashes, unauthorized Host Attacks, intrusions, probes, abuse NIDS/NIPS Failures, DoS, outbound Firewall
    25. 25. Example: OS <ul><li>Account/group changes </li></ul><ul><li>Account logins </li></ul><ul><li>Changes in permissions for critical files/directories </li></ul><ul><li>Shutdowns </li></ul><ul><li>Patches/hotfixes </li></ul><ul><li>Elevated privileges </li></ul>
    26. 26. Example: NIDS and NIPS <ul><li>Intrusion attempts </li></ul><ul><li>Probes </li></ul><ul><li>Admin privilege abuse </li></ul><ul><li>Miscellaneous network anomalies </li></ul><ul><li>AUP violations </li></ul>
    27. 27. Exception vs Audit? <ul><li>Should I log “normal stuff”? </li></ul><ul><ul><li>Firewall deny vs allow </li></ul></ul><ul><ul><li>Resource access </li></ul></ul><ul><li>Alert vs log question </li></ul>
    28. 28. Summary <ul><li>Extensive logging is a must ! </li></ul><ul><ul><li>You now have some hints on what you should log and how to plan </li></ul></ul><ul><li>Monitoring helps extract more value from logs </li></ul><ul><ul><li>And its huge! </li></ul></ul><ul><li>Logging helps with compliance and forensics </li></ul><ul><ul><li>It might even be mandated </li></ul></ul>and …
    29. 29. Q&A? More information? <ul><li>Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA </li></ul><ul><li>anton@chuvakin.org </li></ul><ul><li>Security Strategist </li></ul><ul><li>Author of “Security Warrior” (O’Reilly 2004) – www.securitywarrior.org </li></ul><ul><li>My book on logs is coming soon! </li></ul><ul><li>See www.info-secure.org for my papers, books, reviews and other security resources related to logs </li></ul>