White PaperThe Complete Guide to Log andEvent ManagementDr. Anton Chuvakin
The Complete Guide to Log and Event ManagementTable of Contents:           2      Introduction                            ...
The Complete Guide to Log and Event ManagementIntroduction                                                                ...
The Complete Guide to Log and Event Management•	Searching	is	the	primary	way	to	access	  information in all of the logs, i...
The Complete Guide to Log and Event Management Today’s SIEM tools, such as novell Sentinel,                        log man...
The Complete Guide to Log and Event Management                                                                            ...
The Complete Guide to Log and Event ManagementIf you have logs, you need log management.                          •	Tuning...
The Complete Guide to Log and Event Management        Log	Ignorance:	Logs are not                                         ...
The Complete Guide to Log and Event Managementonce both SIEM and log management have                                Theref...
logging standards and even of logging            Afterward, once both SIEM and logguidance for software developers leads  ...
Upcoming SlideShare
Loading in …5

.The Complete Guide to Log and Event Management


Published on

Security information and event management (SIEM) technology has existed since the late 1990s, but it has always been somewhat controversial in the security industry due to its initial promise of a “security single pane of glass” combined with slow adoption across smaller organizations. More recently, traditional SIEM has been joined by a broaduse log management technology that focuses on collecting a wide variety of logs for a multitude of purposes, from security incident response to regulatory compliance, system management and application troubleshooting. In this paper we will analyze the relationship between these two technologies—SIEM and log management—focusing not only on the technical differences and different uses for these technologies, but also on architecting their joint deployments

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

.The Complete Guide to Log and Event Management

  1. 1. White PaperThe Complete Guide to Log andEvent ManagementDr. Anton Chuvakin
  2. 2. The Complete Guide to Log and Event ManagementTable of Contents: 2 Introduction 3 Security Information and Event Management defining Features 3 Log Management defining Features 4 High-level Comparison: SIEM vs. Log Management 5 SIEM and Log Management Use Cases 6 pCI dSS 6 FISMA 6 HIpAA 6 Technology Trend 7 Example SIEM and Log Management Scenario 7 Architecting Log Management and SIEM 9 What to do First? SIEM or Log Management? 10 do All Companies Have to Graduate from Log Management to SIEM? 11 After Log Management and SIEM: Maturity Curve 13 Mistakes 16 Conclusions 16 About the Author SponSorEd By p. 1
  3. 3. The Complete Guide to Log and Event ManagementIntroduction Security Information and Event well as correlation results to the analysts Management Defining Features in near real time; they can also be fed by Let’s further discuss what features can be historical, archived data. called “defining” SIEM features; most users • Reporting: reporting and scheduled will look for most of these features while reporting covers all the historical views choosing a SIEM product. The features are: of data collected by the SIEM product.Security information and event management novell® Sentinel™, are used by firms large(SIEM) technology has existed since the late and small, from Fortune 1000 or Global • Log and context data collection: This Some products also have a mechanism for1990s, but it has always been somewhat 2000 organizations to tiny SMBs—small and includes being able to collect logs and distributing reports to security personnel orcontroversial in the security industry due to medium businesses. context data (such as identity information IT management, either over e-mail or usingits initial promise of a “security single pane or vulnerability assessment results) using a a dedicated secure Web portal.of glass” combined with slow adoption Before beginning our analysis, it will be helpful combination of agentless and agent-based • Security role workflow: This coversacross smaller organizations. More recently, to define “SIEM” and “log management“and methods. incident management features such astraditional SIEM has been joined by a broad- explain the differences between them. • Normalization and categorization: This being able to open cases and performuse log management technology that focuses covers being able to convert collected investigative tasks, as well as automaticallyon collecting a wide variety of logs for a SIEM covers relevant log collection, original logs into a universal format for use or semi-automatically perform typical tasksmultitude of purposes, from security incident aggregation, normalization and retention; inside the SIEM product. The events are for security operations. Some productsresponse to regulatory compliance, system context data collection; analysis (correlation, also categorized into useful bins such as also include collaborated features thatmanagement and application troubleshooting. prioritization); presentation (reporting, “Configuration Change,” ”File Access” or allow multiple analysts to work on the sameIn this paper we will analyze the relationship visualization); security-related workflow and “Buffer overflow Attack.” security response effort.between these two technologies—SIEM and relevant security content. All the use cases for • Correlation: This is used to describe rule-log management—focusing not only on the SIEM focus on information security, network The above functionality can be found in most based correlation, statistical or algorithmictechnical differences and different uses for security, data security as well as regulatory commercial SIEM products on the market correlation, as well as other methods thatthese technologies, but also on architecting compliance. today. However, most products have strong include relating different events to eachtheir joint deployments. For example, if you and weak points, as well as additional “secret other and events to context data. Correlationneed to satisfy logging requirements of on the other hand, log management sauce” features. could be in real time, but not all toolspCI dSS, which one should you deploy? includes comprehensive log collection, support real-time correlation and insteadWhat technology is better suited to optimize aggregation, original (raw, unmodified) log focus on correlating historical data from theiryour incident response and investigation retention; log text analysis; presentation databases. other log analysis methods areprocedures? Which one will give you real-time (mostly in the form of search, but also Log Management Defining Features sometimes bundled under the correlationinsight about the attacks? In addition, we will reporting); related workflow and content. Let’s start by considering the defining features label as well.provide recommendations for companies With log management, the use cases are of a log management system. These include:that have deployed log management or broad and cover all possible uses for log data • Notification/alerting: This includes being able to trigger notifications or alerts to • Log data collection: This covers beingSIEM in order for them to plot their roadmap across IT and even beyond. operators or managers. Common alerting able to collect all logs using agent-based orto enhancing, optimizing and expanding mechanisms include e-mail, SMS, or even agent-less methods, or a combination of thetheir deployment. We will also recommend The key difference that follows from the above SnMp messages. two.a roadmap for companies that have already definitions stems from the fact that SIEMdeployed both of these technologies. focuses on security—the first word in “security • Prioritization: This includes different • Efficient retention: While collecting and information and event management”—and features that help highlight the important saving log data does not sound like a bigSIEM tools first appeared on the market use of various IT information for security events over less critical security events. engineering challenge, being able to collectin 1997. Their original use was for reducing purposes. on the other hand, log This may be accomplished by correlating gigabytes and even terabytes of log datanetwork intrusion detection system (IdS) “false management focuses on logs and wide- security events with vulnerability data efficiently—and retaining it while providingpositives,” which plagued nIdS systems at ranging uses for log data, both within and or other asset information. prioritization fast searching and quick access to it—is notthe time. The tools were complex to deploy outside the security domain. algorithms would often use severity trivial. Given that many regulations mandateand use, so they were only used by the information provided by the original log specific terms for log data retentionlargest organizations with the most mature source as well. (ranging all the way to multiple years),security programs. The market was sized this functionality is critical to a logat a few million dollars in the late nineties, • Real-time views: This covers security management system.while now, some analysts report that the monitoring dashboards and displays, usedmarket is on track to reach billions in the for security operations personnel. Suchcoming years. Today’s SIEM tools, such as displays will show collected information asp. 2 p. 3
  4. 4. The Complete Guide to Log and Event Management• Searching is the primary way to access information in all of the logs, including can make or break the log management solution. reporting should be fast, now let us review how SIEM and log management technologies are used. recently, traditional SIEM has been logs from custom applications. Search is customizable and easy to use for a broad joined by a broad-use log management indispensable for investigative use of logs, log forensics, and finding faults while using range of purposes. The distinction between searches and reports is pretty clear: Search SIEM and Log Management technology that focuses on collecting a wide logs for application troubleshooting. goes across all available, collected logs in Use Cases variety of logs for a multitude of purposes, A clean and responsive interactive search interface is thus essential for a log raw, original form (like Google goes through Web pages), while report operates on logs Before discussing the joint architecture of from security incident response to regulatory management system. which are parsed into a database (like an SIEM and log management, we need to compliance, system management and briefly present typical use cases that call for• Log indexing or parsing is a key component Excel spreadsheet). Carefully evaluate how easy it is to create a custom report in a log deployment of a SIEM product by a customer application troubleshooting. of a log management system. Indexing can organization. We will start from the very high management tool. This is where a lot of speed up searches literally by a factor of level of three main types of use cases: solutions fall short by requiring that their a hundred. Indexing technology creates a operators study the esoteric aspects of their 1. Security, both detective and investigative: maybe a few hours each day and only review data structure called an index that allows log storage data structures before they can Sometimes also called threat management, alerts and reports as needed and not in very fast keyword type searches and customize the reports. this focuses on detecting and responding near-real time—unless the events happened Boolean type searches across the log to attacks, malware infection, data theft and while they were logged in to the product. storage. Sometimes indexing is used to enable other full text analysis techniques. now let’s perform a high-level comparison other security issues. The third scenario is an “automated SoC” Think about this as “Google for logs.” not between functions and features of SIEM and 2. Compliance, regulatory (global) and policy scenario where an organization configures all log management tools support indexing, log management. (local): This focuses on satisfying the their SIEM to alert based on rules and then or advertise log collection rates that don’t requirement of various laws, mandates “forgets” it until the alert. The analysts never account for indexing, so be careful with and frameworks as well as local corporate log in unless there is a need to investigate vendor claims here. policy. alerts, review reports weekly/monthly or• Reporting and scheduled reporting cover all High-level Comparison: SIEM 3. operational, system and network perform other rare tasks. This is the use case the data collected by the log management vs. Log Management troubleshooting and normal operations: that many smaller organizations want and few product and are similar to SIEM reporting. Specific mostly to log management, this SIEM products can deliver, at least not without In the table below, we show key areas of The strength of reporting, whether for use case has to do with investigating extensive customization. It is worthwhile to functionality and explain how SIEM and log security, compliance or operational reasons, system problems as well as monitoring the add that a lot of SIEM products are sold with management are different. availability of systems and applications. an expectation of being an automated SoC, but such expectations are rarely realized. on a more detailed level, security and compliance use cases fall under several Log management technologies have a role scenarios. Let’s review them in detail. in other scenarios outside of security as well. Functionality Area Security Information and Log Management Application troubleshooting and system Event Management (SIEM) The first usage scenario is a traditional administration are two additional important Log collection Collect security relevant logs Collect all logs including operational Security operations Center (SoC). It typically use cases for log management systems. logs and custom application logs makes heavy use of SIEM features such When the application is deployed and its as real-time views and correlation. A SIEM logging configured, the log management Log retention retain limited parsed and retain raw and parsed log data for customer organization will have analysts system is used to quickly review errors and normalized log data long periods of time online 24x7 and have them “chase” security exception logs. It will also review summaries alerts as they “pop up.” This was the original of normal application activity in order to reporting Security focused reporting, Broad use reporting, historical SIEM use case when SIEM technology started determine application health and troubleshoot real-time reporting reporting in the 1990s; today it is relegated to the possible irregularities. Analysis Correlation, threat scoring, Full text analysis, tagging largest organizations only. event prioritization Another scenario is “compliance status The next use case is sometimes called the reporting.” Here analysts or security Alerting and Advanced security focused Simple alerting on all logs “mini-SoC” scenario. In this case, the security managers review reports with a focus on notification reporting personnel will use non real-time, delayed compliance issues. The review occurs other features Incident management, other High scalability for collection and views to check for security issues (“analysts weekly or monthly or as prescribed by a security data analysis searching come in the morning”). The analysts are online specific regulation. There is not necessarilyp. 4 p. 5
  5. 5. The Complete Guide to Log and Event Management Today’s SIEM tools, such as novell Sentinel, log management controls including the generation, review, protection and retention of While SIEM started as a technology for large global companies and sensitive government They started from their dMZ firewalls and then progressed by feeding additional are used by firms large and small, from audit records, plus steps to take in the event agencies, it continues a march down market. logs into a log management system, whileFortune 1000 or Global 2000 organizations to of audit failure. Many observers predict that 2010 or 2011 will be the year of the major SIEM vendors’ simultaneously defining correlation rules and running reports from the vendor’s pCI dSS tiny SMBs—small and medium businesses. nIST 800-92, “Guide to Computer Security mid-market battle for dominance. As a result, compliance package. As they learned to Log Management,” also created to simplify smaller customers will get much improved respond to alerts, their processes matured FISMA compliance, is fully devoted to log tools for security management. and they started making use of more of the a security or operations focus. This use management. It describes the need for log SIEM functionality. case is commonly a transition phase and management in federal agencies and ways Another trend is acceptance of separate roles the organization will likely later mature to to establish and maintain successful and for SIEM and log management. now, most overall, the project represented a successful one of the aforementioned use cases. Log efficient log management infrastructures— SIEM vendors offer log management solutions implementation of pCI logging requirements. management tools are most often deployed including log generation, analysis, storage as well. This also supports expanding uses The organization passed the pCI assessment for this scenario, but it is not uncommon to and monitoring. nIST 800-92 discusses the for SIEM tools including IT operations, fraud with flying colors and was commended on use a SIEM product for compliance as well. importance of analyzing different kinds of analysis, application troubleshooting, going their comprehensive approach to logging In the latter case, long-term log retention logs from different sources and of clearly all the way up to IT GrC uses for high-level and security monitoring. In addition, the requirements often challenge the deployment. defining specific roles and responsibilities of governance and risk measuring goals. security team built a case that their pCI SIEM those teams and individuals involved in log implementation actually addresses additional Given that logs are very important for meeting management. We’re also witnessing the beginning of compliance mandates since pCI dSS goes compliance mandates, let’s consider a few convergence between IT operations and IT into a deeper level of details while covering regulations in detail. HIPAA management and security management. essentially the same areas of IT governance. The Health Insurance portability and While analysts have predicted this trend for At the same time, log management tools also Accountability Act of 1996 (HIpAA) outlines several years, it has failed to fully materialize bolstered their operational capabilities and PCI DSS relevant security standards for health until now. despite that fact, many predict the overall IT efficiency, while SIEM gave them the The payment Card Industry data Security information. nIST Sp 800-66, “An Introductory trend of convergence of security management core Standard (pCI dSS) applies to organizations resource Guide for Implementing the Health and IT operations management will continue, for their future real-time detection and that handle credit card transactions. Insurance portability and Accountability Act and security tools will have more linkage into response capability. It mandates logging specific details, log Security rule”, details log management IT operational tools such as network and retention and daily log review procedures. requirements for the securing of electronic system management. Even though logging is present in all pCI protected health information. Section 4.1 of Architecting Log Management requirements, pCI dSS also contains nIST 800-66 describes the need for regular and SIEM requirement 10, which is dedicated to logging review of information system activity, such Example SIEM and Log Given the differences between technologies, and log management. Under this requirement, as audit logs, access reports and security Management Scenario many organizations have deployed both SIEM incident-tracking reports. Also, Section 4.22 logs for all system components must be This case study covers a deployment scenario and log management, or are considering specifies that documentation of actions and reviewed at least daily. Further, pCI dSS states of a SIEM and log management solution enhancing an existing deployment of one activities need to be retained for at least six that the organization must ensure the integrity to satisfy pCI-dSS requirements at a large of the technologies with the other. What are years. Logs are sometimes considered part of its logs by implementing file integrity retail chain. The retailer decided to deploy some of the common joint architectures of of that. recent HITECH Act of 2009 promises monitoring and change detection software on a commercial log management solution SIEM and log management? to boost HIpAA implementations in the logs. It also prescribes that logs from in-scope when its pCI assessor suggested it would coming years. systems are stored for at least one year. be required to pass an assessment. A log We will refer to the most common scenario as management vendor suggested that the “SIEM shield.” Many of the organizations that retailer get both log management and SIEM deployed legacy SIEM solutions attempted FISMA solution at the same time. So, it progressed to send too much data to their SIEM, thus Federal Information Security Management Technology Trends from not doing anything with its logs directly overloading it and possibly losing critical Act of 2002 (FISMA) emphasizes the need for As we mentioned before, SIEM technology is to running an advanced log management data and functionality. They addressed each federal agency to develop, document more than 10 years old; it has gone through system and real-time correlation capability. this problem by also acquiring a log and implement an organization-wide program multiple phases which we could write an management tool and deploying it “in front” to secure the information systems that entirely new white paper about. We will The project took a few months following a of their SIEM solution. support its operations and assets. nIST Sp highlight a few of the SIEM technology trends. phased approached. The retailer’s IT staff 800-53, “recommended Security Controls decided to implement it from the outside for Federal Information Systems,” describes in, based on an initial risk assessment. p. 6 p. 7
  6. 6. The Complete Guide to Log and Event Management In the next case, SIEM and log management are deployed alongside each other and at the Being able to respond better has to happen SIEM same time. This is an “emerging scenario” before you are forced to respond faster. since more people now get both at the same time—and typically from the same vendor. Indeed, if an organization somehow realizes It is much easier to be prepared to respond Log Management the need for correlation, it then needs to collect and save all the logs and have the ability to than to monitor. perform efficient search and raw data analytics. obviously, it goes without saying there are lots of “log management only” (still growing) situations and some “SIEM only” (likely shrinking) deployment scenarios.In this case, an inherently more scalable log events are archived on a log management What to Do First? SIEM or Logmanagement tool is deployed in front of SIEM tool. For example, if a total log volume equals Management? SIEM Log Managementto serve as a shield and filter to protect a less 40,000 log messages each second, a SIEM Fortunately, the question of which technologyscalable SIEM tool from extreme log flows. It is tool will receive only 4,000 messages a needs to be deployed first has a verynot uncommon to only send every 10th event second. simple answer. If you have logs, you needreceived by the “log shield” to a SIEM that is log management. This equally applies tohiding behind it. At the same time, all received organizations with one server, all the way to organizations with 100,000 servers. Clearly, the technology they deploy to manage logs will be different, but the existence of logs leads next is a SIEM deployment with log them to log management. For example, if you management as an archive for processed have to review logs from a single machine, and other logs. This scenario arises when SIEM built-in operating system tools will usually somebody buys a big SIEM for security suffice. on the other hand, if your daily log monitoring and then, over time, realizes volume reaches an impressive 100 GB (not Log Management as a Foundation that something is missing. As a result, a log an impossible situation!), sophisticated—and management tool is deployed to “dump” all thus expensive—tools needs to be deployed. logs into and to perform analysis of the raw logs that the SIEM “rejects” (i.e., doesn’t know In fact, even a recent Gartner note “How how to parse, normalize, categorize, etc). ThisAnother scenario emerges when log This is the case where an organization gets to Implement SIEM Technology” (Gartner, leads to a broadening use case from securitymanagement is deployed first to create an a log management tool and slowly realizes 2009) unambiguously states, “deploy log monitoring to incident response and pCI dSSenterprise logging platform. SIEM is then a need—as well as develops an ability—for management functions before you attempt a compliance.added as one of the applications of such a correlation, visualization, monitoring, wide-scale implementation of real-time eventplatform. This scenario can be called “grow workflows, etc. Such a scenario is the most management.” Further, they clarify that whenup to SIEM” and accounts for up to 50 logical for most organizations as we discuss SIEM technology is driven by compliance,percent of SIEM deployments today. further in this paper. the same order of deployment persists: “the first phases of a SIEM deployment that is SIEM primarily driven by pCI would implement log management functions for the systems that are in scope for the pCI assessment.” The overall theme here is that being able to respond better has to happen before you are Log Management forced to respond faster.p. 8 p. 9
  7. 7. The Complete Guide to Log and Event ManagementIf you have logs, you need log management. • Tuning and customization ability: The organization must accept the often enough time for a serious breach to occur, which could take months to clean up. for many other security and IT challenges. This equally applies to organizations responsibility for tuning and customizing the As a result, advanced alerting and stateful At this point, it is worthwhile to note that with one server, all the way to organizations deployed SIEM tool. out-of-the-box SIEM deployments rarely succeed or manage to correlation rules will deliver sub-second responses, but you need to be prepared to some of the log management tools do not offer such a “graduation path” to a SIEM. In with 100,000 servers. reach their full potential. respond to them. particular, simpler tools that only allow you to collect raw logs and perform searches across Let’s review the criteria in detail. In fact, if an organization does not have an them may be extremely useful; however, they What about those organizations that have SoC or any monitoring capability, whether might not allow you an easy way to achieve already deployed legacy SIEM tools? For First, the organization must be ready security monitoring or operational monitoring full normalization, categorization and other them, looking into log management as soon to respond to alerts soon after they are with strict SLAs, many of the SIEM features security-focused enrichment of log data. In as possible is a smart thing to do. Being produced. While the claims that “modern will not be fully utilized. A common first step general, if your tool collects and retains raw able to go through a complete collection business works in real-time and so the from purely responsive use of logs to full- log records and cannot be paired with a of log records will boost their investigative security should too” are often heard blown security monitoring is utilizing “delayed SIEM solution that can make such data for capabilities and help them meet compliance from various vendors, it appears that few periodic monitoring” which really means security monitoring and analysis, graduation mandates. organizations are able to achieve that at the “reviewing log reports every morning.” This to monitoring will not be possible. other tools moment. So, before deploying SIEM ask: How can be accomplished with a log management will need to be purchased if your organization Do All Companies Have to Graduate real-time is your security? one might think that tool or with a SIEM tool. becomes ready for real-time monitoring. from Log Management to SIEM? most of the time, security is indeed in real-time What happens after an organization deploys or very close to it. network intrusion detection The final graduation criteria relates to tuning Given that using a SIEM solution effectively a log management tool and starts using it systems pick up attacks off the wire within and customization ability. The organization gives you direct threat reduction benefits via effectively for security and compliance as well microseconds, firewalls block connections as must accept the responsibility for tuning its advanced security focused analysis (but as operational purposes? The natural and they happen, and anti-virus technology makes and customizing the deployed SIEM tool in only if your organization is ready for SIEM), the logical progression is for organizations to the best effort to catch the viruses as soon as order to fit its powerful and customizable “compliance+” model makes sense. overall, graduate to near-real-time event management they arrive. features to a problem set that an organization it allows the organization to move closer by deploying a SIEM tool. faces. A second option is to hire a specialist to that mythical “single-pane of glass” for Thus, few people will agree to buy a network consulting firm to do the tuning for them. security management. This paper is the first document that intrusion detection system (nIdS) that will Every business is unique, and in order to formulates “graduation criteria” for such only notify of an attack after two have passed. be most effective, a SIEM must take into After Log Management and SIEM: development. organizations that graduate However, those same people will have their account the unique business processes that Maturity Curve too soon will waste time and effort, and security analysts check the IdS alarms exist. This might mean creating alerts, writing What happens next after both log won’t realize any increased efficiency in their every morning. If they discover a critical correlation rules or customizing reports in management and SIEM are deployed and security operation. However, waiting too long compromise, a millisecond response time of order to gain insight about the organization’s “operationalized” to help with compliance and also means that the organization will never the nIdS system will not matter, but the hourly security or compliance posture. From the deliver security benefits to an organization? develop the necessary capabilities to secure response time of the personnel will. So, if the author’s experience, it is worthwhile to note There is a maturity curve that stretches from themselves. “morning after” alert investigation results in that out-of-the-box deployments with inflated complete log ignorance, to log collection discovering a critical system compromise, it is expectation of SIEM as “analyst-in-the-box” and retention, to occasional investigation, to In brief, the criteria are: still deemed acceptable. rarely succeed. periodic log review and then all the way to near-real-time security monitoring. • Response capability: The organization Similarly, if a virus-infected file arrives and What is interesting is that organizations that must be ready to respond to alerts soon the software can clean it “in real-time”, the have no immediate plans to migrate from, say, The trend here is from being ignorant, to being after they are produced. problem is solved. However, in case the compliance-focused log management should slowly reactive, to being quickly reactive, to • Monitoring capability: The organization antivirus software detects the malicious code, still choose a logging tool that allows them eventually being proactive and aware of what must have or start to build security but cannot automatically clean or quarantine to later graduate to SIEM. Even with no initial is going on across your IT environment. Trying monitoring capability by creating a Security it and issues an alert instead (which happens plans to move beyond compliance, many to make one jump from ignorant to proactive operation Center (SoC) or at least a team in the case of some backdoors and Trojans), SIEM and log management deployments rarely, if ever, works! dedicated to ongoing periodic monitoring. the response falls back on the shoulders follow so-called “compliance+” models, of the analysts who are likely hours behind. which means that the tool is purchased for a With today’s sophisticated threats, this is particular regulatory framework, but is utilized p. 10 p. 11
  8. 8. The Complete Guide to Log and Event Management Log Ignorance: Logs are not Logs: Activities, Actions, Events collected or reviewed. Security Information and Event Log Collection: Logs are collected Vulnerability Assessment Data and stored, but never looked at. Management Log Investigation: Logs are collected and looked at in case of an incident. Identity and Log Reporting: Logs are collected and Access Management reports are reviewed every month. Users: Identities, Roles, Rights Log Review: Logs are collected and reviewed daily (delayed monitoring). Log Monitoring: Security information is monitored in near-real time. In addition, an asset management system Mistakes will contain similar detailed information on all When planning and implementing log IT resources within the organization. Just like collection and analysis infrastructure— we can do with users, we can extract asset whether for SIEM or log management—the business role, business criticality, compliance organizations often discover that they are notWhat is the next step in the evolution after that examples involves using information from relevance, administrator name and location as realizing the full promise of such systems. Inpoint? For starters, organizations should be identity management systems such as novell well as other information on what function the fact, they sometimes notice that efficiencycontinuously improving the breadth and depth Identity Manager. The information available asset performs and who is responsible for it. is not gained, but is lost as a result. Thisof SIEM deployment by integrating it with in this system includes user identity (such as Such information will dramatically improve risk often happens due to the following commonmore systems to make better use of SIEM’s real name, work role, business unit affiliation, computation and event prioritization functions implementation mistakes.analytics capabilities. This gets at SIEM’s etc.) as well as access rights across various of SIEM. Be aware that even though manycore mission—security monitoring—and systems and applications. Knowing who vendors claim identity integration, most will We will start from the obvious—butalso solves new problems such as fraud, the user is and what he is allowed to do is only perform a simple LdAp lookup. These unfortunately all too common—mistake, eveninsider threat, application monitoring and indispensable for security monitoring of insider systems lose out on the all the rich data an in this age of Sarbanes-oxley and pCI dSS.overall user activity monitoring. SIEM starts activities. For example, it allows you to create identity system could provide to help a SIEM This mistake destroys all possible chances ofto acquire more information and to move “a unified identity” for each user and then determine if activities are malicious or have benefiting from log management or SIEM.up the stack from network to application, use it to monitor user actions across multiple regulatory relevance.from a limited number of data sources to systems, even with different user names and The first mistake is not logging at all. Anotherenterprise-wide deployment. At the same accounts. Further levels of integration—and thus version of the same mistake is not loggingtime, a security organization grows with it and increased awareness—can be provided by and not even knowing it until it is too late.develops better operational procedures that on top of this, identity manager integration integrating with configuration managementallow the organization to be more agile. While allows a SIEM product to differentiate databases (CMdB). Such integrations allow a How can it be too late? not having logs canexpanding the deployment, it is crucial to authorized, official logins from backdoor, SIEM product to correlate detected changes lead to losing your income (pCI dSS loggingremember that a phased approach is the only unauthorized login attempts. Such integration across systems and applications with requirements imply that violations might leadway to succeed here. also allows automated separation-of-duty approved and authorized changes. to your credit card processing privileges being (Sod) monitoring by making SIEM aware canceled by Visa or MasterCard, thus puttingWhat are some of the systems that would of which roles are not allowed to perform you out of business), reputation (somebodyenhance SIEM’s mission and allow it to solve specific actions. stole a few credit card numbers from yourother problems? one of the most interesting database, but the media reported that all of the 40 million credit cards have been stolen since you were unable to prove otherwise) or even your freedom (see various Sarbanes- oxley horror stories in the media).p. 12 p. 13
  9. 9. The Complete Guide to Log and Event Managementonce both SIEM and log management have Therefore, once the technology is in place and logs are collected, there must be a process of and investigation or troubleshooting. This leads to the horrible realization after the For example, many people would claim that network intrusion detection and prevention been operationalized, your organization ongoing monitoring and review that hooks into incident that all logs are gone due to their logs are inherently more important than, say, can move up the maturity scale to actions and possible escalations, if needed. In addition, personnel reviewing or monitoring shortsighted retention policy. It often happens (especially in the case of insider attacks) Vpn concentrator logs. Well, it might be true in the world where external threats completely comprehensive network and application logs should have enough information to that the incident is discovered a long time— dominate the insider abuse and all employees visibility, user activity monitoring and other determine what they really mean and what—if any—action is required. sometimes many months—after the crime or abuse has been committed. one might save and partners can simply be trusted. Vpn logs, together with server and workstation logs, are integration with different systems. some money on storage hardware, but lose it what you would most likely need to conduct It is worthwhile to note that some tenfold due to regulatory fines. an internal investigation about the information organizations take a half step in the right leak or even a malware infection. Thus, similar Even organizations that are well-prepared fall direction: They only review logs (provided If low cost is critical, the solution is sometimes claims about the elevated importance of for this mistake. Consider this recent example. they didn’t commit the first mistake and to split the retention in two parts: short-term any other log type can be similarly disputed, does your Web server have logging enabled? they actually have something to review) online storage (that costs more) and long-term which would lead us to a painful realization Sure, it is a default option on both of the after a major incident (be it a compromise, offline storage (that is much cheaper). A good that you do need to collect everything or most popular Web servers: Apache and Microsoft information leak or a mysterious server log management tool will allow you to search of the log records produced. But can you? IIS. does your server operating system log crash) and avoid ongoing monitoring and log through both of these stores transparently, Before you answer this, try to answer whether messages? Sure, nobody canceled /var/log/ review, often by quoting the proverbial lack of without moving data around. A better three- you can make the right call on which log is messages. But does your database? The resources. This gives them the reactive benefit tier approach is also common and resolves more important even before seeing it and this default option in oracle is to not perform any of log analysis, which is important, but fails to some of the limitations of the previous one. problem will stop looking unsolvable. In fact, data access audit logging. does Microsoft realize the proactive one: knowing when bad In this case, shorter-term online storage is there are cost-effective solutions to achieve SQL fare better? Sadly, the answer is “no”, you stuff is about to happen or become worse. For complemented by a near-line storage where just that. need to dig deep in the system to even start a example, if you review logs, you might learn logs are still accessible and searchable. The moderate level of audit trail generation. that the failover was activated on a firewall, oldest and the least relevant log records are The way to avoid this mistake is to deploy log and, even though the connection stayed on, offloaded to the third tier, such as tape or management before SIEM as we prescribe Thus, to avoid this mistake one needs to the incident is certainly worth looking into. If dVds, where they can be stored inexpensively. earlier. This will guarantee that all needed sometimes go beyond the defaults and make you don’t and your network connectivity goes However, there is no way to selectively access logs are available for analysis, even if only a sure that the software and hardware deployed away, you’d have to rely on your ever-helpful the needed logs. More specifically, one percentage is ever seen by a SIEM does have some level of logging enabled. In logs to investigate why both failover devices financial institution was storing logs online correlation engine. case of oracle, for example, it might boil down went down. for 90 days, then in the near-line searchable to making sure that the “audit_trail” variable storage of the log management system for The final mistake is ignoring the logs is set to “db.” For other systems it might be It is also critical to stress that some types of two years, and then on tape for up to seven from applications, by only focusing on the more complicated. organizations have to look at log files and years or even more in some cases. perimeter and internal network devices, and audit tracks due to regulatory pressure of possibly also servers, but not going higher up Not reviewing logs is the second mistake. some kind. As we mention previously, HIpAA The fourth mistake is related to log record the stack to look at the application logging. While making sure that logs do exist and then regulation compels medical organizations prioritization. While people need a sense collecting and storing them is important, it to establish an audit record and analysis of priority to better organize their log The realm of enterprise applications is only a means to an end: knowing what is program (even though the enforcement action analysis efforts, the common mistake today ranges from SAp and peopleSoft to small going on in your environment and being able is notoriously lacking). Also, pCI dSS data is prioritizing the log records before homegrown applications, which nevertheless to respond to it, as well as possibly predict security standard has provisions for both log collection. In fact, even some “best practice” handle mission-critical processes for many what will happen later. As we describe above, collection and log monitoring and periodic documents recommend only collecting “the enterprises. Legacy applications, running on it is a stage, but not the destination. If your review, highlighting the fact that collection of important stuff.” But what is important? This mainframes and midrange systems, are out company has just moved from ignoring logs logs does not stand on its own. is where the above guidance documents fall there as well, often running the core business to collecting logs, it is important to know that short by not specifying it in any useful form. processes too. The availability and quality ultimately you will need to review them. If you The third common mistake is storing logs While there are some approaches to the of logs differ wildly across the application, collect logs and don’t review them, you are for too short a time. A SIEM system’s problem, it can lead to glaring holes in security ranging from missing (the case for many simply documenting your own negligence, operational log store might retain normalized posture or even undermine the regulatory home-grown applications) to extremely especially if your IT security policy prescribes events for 30 days, but a log management compliance efforts. detailed and voluminous (the case for many log reviews. system is needed for long term retention. This mainframe applications). Lack of common makes the security or IT operations team think they have all the logs needed for monitoring p. 14 p. 15
  10. 10. logging standards and even of logging Afterward, once both SIEM and logguidance for software developers leads management have been operationalized,to many challenges with application logs. your organization can move up the maturityFortunately, future efforts such as MITrE CEE scale to comprehensive network andwill remediate this problem. application visibility, user activity monitoring and other integration with different systems.despite the challenges, you need to makesure that the application logs are collectedand made available for analysis as wellas for longer term retention. This can be About the Authoraccomplished by configuring your logmanagement software to collect them and dr. Anton Chuvakin (http://www.chuvakin.by establishing a log review policy, both for org) is a recognized security expert in thethe on-incident review and periodic proactive field of log management and pCI dSSlog review. Look for vendors that make it compliance. He is the author of two bookseasy to configure their systems to collect “Security Warrior” and “pCI Compliance”logs from custom applications, as these and a contributor to “Know your Enemyare often the most important. Later you II”, “Information Security Managementcan configure SIEM to analyze the logs for Handbook” and others. Anton has publishedsecurity purposes, together with network and dozens of papers on log management,other logs. correlation, data analysis, pCI dSS, security management (see a list at www.info-secure. org). His blog http://www.securitywarrior.org is one of the most popular in the industry. In addition, Anton teaches classes andConclusions presents at many security conferencesone of the paramount conclusions from this across the world; he recently addressedwork is to remember that everybody has logs audiences in United States, UK, Singapore,and that means that everybody ultimately Spain, russia and other countries. He worksneeds log management. In its broadest form, on emerging security standards and serveslog management simply means “dealing on the advisory boards of several securitywith logs.” And if you have logs, you have to start-ups.deal with them—if only because many recentregulatory mandates prescribe that. Currently, Anton is developing his security consulting practiceIt’s also important to remember that logs are www.securitywarriorconsulting.com, focusingused for a very large number of situations: on logging and pCI dSS compliancefrom traditional (incident response) to highly for security vendors and Fortune 500esoteric. Most uses of logs happen much organizations. dr. Anton Chuvakin waslater, after the event happens and is recorded formerly a director of pCI Compliancein logs. It is much easier to be prepared to Solutions at Qualys. previously, Antonrespond than to monitor. worked at LogLogic as a Chief Logging Evangelist, tasked with educating theyour organization might need to go “back world about the importance of logging forto logging school” before it is ready to security, compliance and operations. Before“graduate to SIEM.” Such graduation LogLogic, Anton was employed by a securityrequires an ability to respond to alerts and vendor in a strategic product managementcustomize and tune products. role. Anton earned his ph.d. from Stony Brook University.p. 16