This white paper presents Beehive, a novel system that attacks the problem of automatically mining and extracting knowledge from the dirty log data produced by a wide variety of security products in a large enterprise.
This paper describes the concept of implementing the network vulnerability assessment process as a web service in Eucalyptus cloud.This paper is published in one of the international conferences.I implemented the mentioned concept during my M.E. thesis.
Join CTO and Nonprofit Cybersecurity expert Matthew Eshleman as he walks through the third annual Community IT Nonprofit Cybersecurity Incident Report.
This report looks at the different types of attacks that occur at small and mid-sized nonprofit organizations. Is your nonprofit prepared?
Matt also shares advice on security improvements that provide protection against the most common attacks. Learn the role of leadership in placing a value on cybersecurity preparedness for your nonprofit and the long term planning that should accompany your immediate assessment of your security risk.
Matt touches on vendor hacks from 2020 including Blackbaud and SolarWinds and discusses steps your nonprofit should take to understand your risk level.
Learn about real cyberattacks on nonprofit organizations and how they responded to these attempted hacks. Matt gives you the tools you need to protect your organization and staff from cybercrimes.
Many of these tips you can put in place quickly and train your staff on immediately.
Download the full report or view here: https://communityit.com/2021-nonprofit-cybersecurity-incident-download/
This presentation delves into the many cybersecurty risks that plague the healthcare industry and how these risks can be mitigated with the help of security solutions that Seqrite offers.
The Avid Life Media hack is a striking example of everything that can go wrong when a company is completely breached followed by a total disclosure of the stolen information. This attack resulted in an estimated $200 million in costs, firing of the CEO, and countless lives ruined. This presentation will review the data exposed and what can be learned to prevent this from happening to your organization.
This paper describes the concept of implementing the network vulnerability assessment process as a web service in Eucalyptus cloud.This paper is published in one of the international conferences.I implemented the mentioned concept during my M.E. thesis.
Join CTO and Nonprofit Cybersecurity expert Matthew Eshleman as he walks through the third annual Community IT Nonprofit Cybersecurity Incident Report.
This report looks at the different types of attacks that occur at small and mid-sized nonprofit organizations. Is your nonprofit prepared?
Matt also shares advice on security improvements that provide protection against the most common attacks. Learn the role of leadership in placing a value on cybersecurity preparedness for your nonprofit and the long term planning that should accompany your immediate assessment of your security risk.
Matt touches on vendor hacks from 2020 including Blackbaud and SolarWinds and discusses steps your nonprofit should take to understand your risk level.
Learn about real cyberattacks on nonprofit organizations and how they responded to these attempted hacks. Matt gives you the tools you need to protect your organization and staff from cybercrimes.
Many of these tips you can put in place quickly and train your staff on immediately.
Download the full report or view here: https://communityit.com/2021-nonprofit-cybersecurity-incident-download/
This presentation delves into the many cybersecurty risks that plague the healthcare industry and how these risks can be mitigated with the help of security solutions that Seqrite offers.
The Avid Life Media hack is a striking example of everything that can go wrong when a company is completely breached followed by a total disclosure of the stolen information. This attack resulted in an estimated $200 million in costs, firing of the CEO, and countless lives ruined. This presentation will review the data exposed and what can be learned to prevent this from happening to your organization.
Learn the unique capabilities of using indicators to deal with the security breaches and attacks in the security information and event management space. Detect and enrich the indicators using ManageEngine Log360.
In this presentation we will look at the cause and effect of the problem, analyze preparedness and learn how you can better prepare, detect, respond and recover from cyber-attacks.
Insider threats come in a variety of forms and may be malicious or simply the result of negligence. Insider attacks can cause more damage than outsider threats, so it is important that organizations understand how to protect against and remedy insider threats. Learn more about insider threats and GTRI's Insider Threat Security Solution in this presentation. (Source: GTRI)
This presentation includes information about Cisco Stealthwatch, which goes beyond conventional threat detection and harnesses the power of NetFlow. With it, you get advanced network visibility, analytics, and protection. You see everything happening across your network and data center. And you can uncover attacks that bypass the perimeter and infiltrate your internal environment. (Source: Cisco)
Application security is the use of hardware, software and procedural methods in order to protect applications from internal or external threats. As more and more applications are becoming accessible over networks, they are being exposed to a wide variety of threats as well.
Crush Common Cybersecurity Threats with Privilege Access ManagementBeyondTrust
In this presentation from his webinar, IoT Security Expert Rob Black, CISSP, Founder and Managing Principal of Fractional CISO, discusses the common thread of many of today's cyberattacks. Key themes covered include:
- Post-mortem analysis of recent cybersecurity attacks and how you could mitigate against similar threats
- Evaluation of password breakdowns in protecting your organization
- Review of a high level threat model of privileged accounts
- How Privilege Access Management can significantly reduce your attack surface and improve your cybersecurity posture
The Next Generation Cognitive Security Operations Center: Network Flow Forens...Konstantinos Demertzis
A Security Operations Center (SOC) can be defined as an organized and highly skilled team that uses advanced computer forensics tools to prevent, detect and respond to cybersecurity incidents of an organization. The fundamental aspects of an effective SOC is related to the ability to examine and analyze the vast number of data flows and to correlate several other types of events from a cybersecurity perception. The supervision and categorization of network flow is an essential process not only for the scheduling, management, and regulation of the network’s services, but also for attacks identification and for the consequent forensics’ investigations. A serious potential disadvantage of the traditional software solutions used today for computer network monitoring, and specifically for the instances of effective categorization of the encrypted or obfuscated network flow, which enforces the rebuilding of messages packets in sophisticated underlying protocols, is the requirements of computational resources. In addition, an additional significant inability of these software packages is they create high false positive rates because they are deprived of accurate predicting mechanisms.
For all the reasons above, in most cases, the traditional software fails completely to recognize unidentified vulnerabilities and zero-day exploitations. This paper proposes a novel intelligence driven Network Flow Forensics Framework (NF3) which uses low utilization of computing power and resources, for the Next Generation Cognitive Computing SOC (NGC2SOC) that rely solely on advanced fully automated intelligence methods. It is an effective and accurate Ensemble Machine Learning forensics tool to Network Traffic Analysis, Demystification of Malware Traffic and Encrypted Traffic Identification.
Top Solutions and Tools to Prevent Devastating Malware White PaperNetIQ
Custom malware frequently goes undetected. According to Forrester Research, the best way to reduce risk of breach is to deploy file integrity monitoring (FIM) tools that provide immediate alerts.
Intrusion detection and anomaly detection system using sequential pattern miningeSAT Journals
Abstract
Nowadays the security methods from password protected access up to firewalls which are used to secure the data as well as the networks from attackers. Several times these types of security methods are not enough to protect data. We can consider the use of Intrusion Detection Systems (IDS) is the one way to secure the data on critical systems. Most of the research work is going on the effectiveness and exactness of the intrusion detection, but these attempts are for the detection of the intrusions at the operating system and network level only. It is unable to detect the unexpected behavior of systems due to malicious transactions in databases. The method used for spotting any interferes on the information in the form of database known as database intrusion detection. It relies on enlisting the execution of a transaction. After that, if the recognized pattern is aside from those regular patterns actual is considered as an intrusion. But the identified problem with this process is that the accuracy algorithm which is used may not identify entire patterns. This type of challenges can affect in two ways. 1) Missing of the database with regular patterns. 2) The detection process neglects some new patterns. Therefore we proposed sequential data mining method by using new Modified Apriori Algorithm. The algorithm upturns the accurateness and rate of pattern detection by the process. The Apriori algorithm with modifications is used in the proposed model.
Keywords — Anomaly Detection, Modified Apriori Algorithm, Misuse detection, Sequential Pattern Mining
One of the most critical aspects of safeguarding the IT assets of any corporation is dealing with the Insider's Threat. With so many diversified IT components, it is a real challenge to design an effective IT security strategy. It is critical to recognize this particular threat and take countermeasures to protect your assets. So, this webinar covers: Insider threats, how to mitigate insider threats, how to design an effective IT security strategy, and how to protect your assets.
Main points covered:
• Insider threats
• How to design an effective IT security strategy
• How to protect your assets
Presenter:
The webinar was hosted by Demetris Kachulis. Mr. Kachulis is an expert in the field of Information Security. With over 20 years of Wall Street consulting experience, he has worked with many Fortune 500 companies. He is currently the director of Eldion Consulting, a company offering Security, Trainings and Business solutions.
Link of the recorded session published on YouTube: https://youtu.be/hXe5HHjnBeU
Ethical Hacking Concepts and Scopes, Threats and Attack Vectors, Information Assurance, Threat Modelling
Enterprise Information Security Architecture, Vulnerability
Assessment and Penetration Testing
Types of Social Engineering, Insider Attack, Preventing Insider
Threats, Social Engineering Targets and Defence Strategies
Learn the unique capabilities of using indicators to deal with the security breaches and attacks in the security information and event management space. Detect and enrich the indicators using ManageEngine Log360.
In this presentation we will look at the cause and effect of the problem, analyze preparedness and learn how you can better prepare, detect, respond and recover from cyber-attacks.
Insider threats come in a variety of forms and may be malicious or simply the result of negligence. Insider attacks can cause more damage than outsider threats, so it is important that organizations understand how to protect against and remedy insider threats. Learn more about insider threats and GTRI's Insider Threat Security Solution in this presentation. (Source: GTRI)
This presentation includes information about Cisco Stealthwatch, which goes beyond conventional threat detection and harnesses the power of NetFlow. With it, you get advanced network visibility, analytics, and protection. You see everything happening across your network and data center. And you can uncover attacks that bypass the perimeter and infiltrate your internal environment. (Source: Cisco)
Application security is the use of hardware, software and procedural methods in order to protect applications from internal or external threats. As more and more applications are becoming accessible over networks, they are being exposed to a wide variety of threats as well.
Crush Common Cybersecurity Threats with Privilege Access ManagementBeyondTrust
In this presentation from his webinar, IoT Security Expert Rob Black, CISSP, Founder and Managing Principal of Fractional CISO, discusses the common thread of many of today's cyberattacks. Key themes covered include:
- Post-mortem analysis of recent cybersecurity attacks and how you could mitigate against similar threats
- Evaluation of password breakdowns in protecting your organization
- Review of a high level threat model of privileged accounts
- How Privilege Access Management can significantly reduce your attack surface and improve your cybersecurity posture
The Next Generation Cognitive Security Operations Center: Network Flow Forens...Konstantinos Demertzis
A Security Operations Center (SOC) can be defined as an organized and highly skilled team that uses advanced computer forensics tools to prevent, detect and respond to cybersecurity incidents of an organization. The fundamental aspects of an effective SOC is related to the ability to examine and analyze the vast number of data flows and to correlate several other types of events from a cybersecurity perception. The supervision and categorization of network flow is an essential process not only for the scheduling, management, and regulation of the network’s services, but also for attacks identification and for the consequent forensics’ investigations. A serious potential disadvantage of the traditional software solutions used today for computer network monitoring, and specifically for the instances of effective categorization of the encrypted or obfuscated network flow, which enforces the rebuilding of messages packets in sophisticated underlying protocols, is the requirements of computational resources. In addition, an additional significant inability of these software packages is they create high false positive rates because they are deprived of accurate predicting mechanisms.
For all the reasons above, in most cases, the traditional software fails completely to recognize unidentified vulnerabilities and zero-day exploitations. This paper proposes a novel intelligence driven Network Flow Forensics Framework (NF3) which uses low utilization of computing power and resources, for the Next Generation Cognitive Computing SOC (NGC2SOC) that rely solely on advanced fully automated intelligence methods. It is an effective and accurate Ensemble Machine Learning forensics tool to Network Traffic Analysis, Demystification of Malware Traffic and Encrypted Traffic Identification.
Top Solutions and Tools to Prevent Devastating Malware White PaperNetIQ
Custom malware frequently goes undetected. According to Forrester Research, the best way to reduce risk of breach is to deploy file integrity monitoring (FIM) tools that provide immediate alerts.
Intrusion detection and anomaly detection system using sequential pattern miningeSAT Journals
Abstract
Nowadays the security methods from password protected access up to firewalls which are used to secure the data as well as the networks from attackers. Several times these types of security methods are not enough to protect data. We can consider the use of Intrusion Detection Systems (IDS) is the one way to secure the data on critical systems. Most of the research work is going on the effectiveness and exactness of the intrusion detection, but these attempts are for the detection of the intrusions at the operating system and network level only. It is unable to detect the unexpected behavior of systems due to malicious transactions in databases. The method used for spotting any interferes on the information in the form of database known as database intrusion detection. It relies on enlisting the execution of a transaction. After that, if the recognized pattern is aside from those regular patterns actual is considered as an intrusion. But the identified problem with this process is that the accuracy algorithm which is used may not identify entire patterns. This type of challenges can affect in two ways. 1) Missing of the database with regular patterns. 2) The detection process neglects some new patterns. Therefore we proposed sequential data mining method by using new Modified Apriori Algorithm. The algorithm upturns the accurateness and rate of pattern detection by the process. The Apriori algorithm with modifications is used in the proposed model.
Keywords — Anomaly Detection, Modified Apriori Algorithm, Misuse detection, Sequential Pattern Mining
One of the most critical aspects of safeguarding the IT assets of any corporation is dealing with the Insider's Threat. With so many diversified IT components, it is a real challenge to design an effective IT security strategy. It is critical to recognize this particular threat and take countermeasures to protect your assets. So, this webinar covers: Insider threats, how to mitigate insider threats, how to design an effective IT security strategy, and how to protect your assets.
Main points covered:
• Insider threats
• How to design an effective IT security strategy
• How to protect your assets
Presenter:
The webinar was hosted by Demetris Kachulis. Mr. Kachulis is an expert in the field of Information Security. With over 20 years of Wall Street consulting experience, he has worked with many Fortune 500 companies. He is currently the director of Eldion Consulting, a company offering Security, Trainings and Business solutions.
Link of the recorded session published on YouTube: https://youtu.be/hXe5HHjnBeU
Ethical Hacking Concepts and Scopes, Threats and Attack Vectors, Information Assurance, Threat Modelling
Enterprise Information Security Architecture, Vulnerability
Assessment and Penetration Testing
Types of Social Engineering, Insider Attack, Preventing Insider
Threats, Social Engineering Targets and Defence Strategies
Understanding Its Suspicious Activity Reporting (SAR) Requirementcomplianceonline123
What is Suspicious Activity Report (SAR)?
BSA requires every US national bank to file a Suspicious Activity Report (SAR) when they detect certain known or suspected violations of federal law or suspicious transactions related to a money laundering activity or a violation of the BSA.
Purpose of SAR:
Identify new methodologies of money-laundering
Offer data for law enforcement investigation
Deter and Constraint money-laundering
BSA requires every US national bank to file a Suspicious Activity Report (SAR) when they detect certain known or suspected violations of federal law or suspicious transactions related to a money laundering activity or a violation of the BSA.
Purpose of SAR:
Identify new methodologies of money-laundering
Offer data for law enforcement investigation
Deter and Constraint money-laundering
Distinction between CTR and SAR
CTR is required whenever the transaction or series of transactions exceeds the minimum threshold requirement in a 24 hour period.
SAR is required to be filed when there are elements of uselessness, suspicion or indicators of potential illegal activity. Minimum threshold requirement is not applicable to SAR situation.
How to Identify Suspicious Activity?
Banks can use a number of methods to track and identify unusual activity – this may include:
Employee identification
Law enforcement enquiries and requests
Transaction and surveillance monitoring system output
Any combination of the above
When is SAR Filling Required?
A SAR filing is required for any potential crimes:
involving insider abuse regardless of the dollar amount;
where there is an identifiable suspect and the transaction involves $5,000 or more; and
where there is no identifiable suspect and the transaction involves $25,000 or more
When to File SARs:
A SAR should be filed no later than 30 calendar days from the date of the initial detection of facts that may constitute a basis for filing a SAR.
In cases where no suspect can be identified, the time period for filing a SAR is extended to 60 days.
SAR Narratives
SAR Narratives Should:
Be concise and clear
Be chronological and complete
Provide a detailed description of the known or suspected criminal violation or suspicious activity
Identify the essential elements of the information
Outline of Effective SAR Narrative:
Introduction
Body
Conclusion
Want to learn more about anti-money laundering process, BSA requirements and best practices? ComplianceOnline webinars and seminars are a great training resource. Check out the following links:
How to Write an Effective SAR Narratives
Best Practices for Writing Effective SAR.
Managing an Effective AML Compliance Program
Are You Doing Your BSA/AML Risk Assessment Properly?
How to Report under AML/BSA?
BSA/AML Compliance Checklist
How to Create Effective AML/BSA Compliance Program?
How to Develop Risk Models for AML Monitoring Program?
presentatie voor de campaign 'let your blue jeans talk ... green'. Ecologische en sociale impact van kledij, meer bepaald jeans. Het hele lessenpakket is te downloaden op http://greenjeans.be/index.php?option=com_content&view=article&id=55&Itemid=37
The Banker Special Report: Moving Tech with the TimesEMC
This 13-page supplement to the December 2012 issue of The Banker presents EMC Consulting's 2013 perspective of how IT Transformation enables improved capabilities around Big Data, trust, and multi-channel sales and service.
This white paper discusses the various cyber threats targeting healthcare organizations and the challenges security professionals face in securing access to protected health information.
Deep Learning based Threat / Intrusion detection systemAffine Analytics
The article is about a Threat/Intrusion Detection System, which could be used to detect such data leaks/breaches & take a preventive action to contain, if not stop the damage due to breach.
Insight Brief: Security Analytics to Identify the 12 Indicators of Compromise21CT Inc.
In this security insight brief, 21CT researchers look at the malicious network behaviors that concern organizations the most, and how to use security analytics to find them before damage is done. Understanding these 12 indicators of compromise are critical to identifying a network breach.
Running Head Security Assessment Repot (SAR) .docxSUBHI7
Running Head: Security Assessment Repot (SAR) 1
Security Assessment Report (SAR) 27
Intentionally left blank
Security Assessment Report (SAR)
CHOICE OF ORGANIZATION IS UNIVERSITY OF MARYLAND MEDICAL CENTER (UMMC) OR A FICTITIUOS ORGANIZATION (BE CREATIVE)
Introduction
· Research into OPM security breach.
· What prompts this assessment exercise in our choice of organization? “but we have a bit of an emergency. There's been a security breach at the Office of Personnel Management. need to make sure it doesn't happen again.
· What were the hackers able to do? OPM OIG report and found that the hackers were able to gain access through compromised credentials
· How could it have been averted? A) security breach could have been prevented, if the Office of Personnel Management, or OPM, had abided by previous auditing reports and security findings.b) access to the databases could have been prevented by implementing various encryption schemas and c) could have been identified after running regularly scheduled scans of the systems.
Organization
· Describe the background of your organization, including the purpose, organizational structure,
· Diagram of the network system that includes LAN, WAN, and systems (use the OPM systems model of LAN side networks), the intra-network, and WAN side networks, the inter-net.
· Identify the boundaries that separate the inner networks from the outside networks.
· include a description of how these platforms are implemented in your organization: common computing platforms, cloud computing, distributed computing, centralized computing, secure programming fundamentals (cite references)
Threats Identification
Start Reading: Impact of Threats
The main threats to information system (IS) security are physical events such as natural disasters, employees and consultants, suppliers and vendors, e-mail attachments and viruses, and intruders.
Physical events such as fires, earthquakes, and hurricanes can cause damage to IT systems. The cost of this damage is not restricted to the costs of repairs or new hardware and software. Even a seemingly simple incident such as a short circuit can have a ripple effect and cost thousands of dollars in lost earnings.
Employees and consultants; In terms of severity of impact, employees and consultants working within the organization can cause the worst damage. Insiders have the most detailed knowledge of how the information systems are being used. They know what data is valuable and how to get it without creating tracks.
Suppliers and vendors; Organizations cannot avoid exchanging information with vendors, suppliers, business partners, and customers. However, the granting of access rights to any IS or network, if not done at the proper level—that is, at the least level of privilege—can leave the IS or ne ...
Include at least 250 words in your posting and at least 250 words inmaribethy2y
Include at least 250 words in your posting and at least 250 words in your reply. Indicate at least one source or reference in your original post. Please see syllabus for details on submission requirements.
Module 1 Discussion Question
Search "scholar.google.com" for a company, school, or person that has been the target of a network
or system intrusion? What information was targeted? Was the attack successful? If so, what changes
were made to ensure that this vulnerability was controlled? If not, what mechanisms were in-place to protect against the intrusion.
Reply-1(Shravan)
Introduction:
Interruption location frameworks (IDSs) are programming or equipment frameworks that robotize the way toward observing the occasions happening in a PC framework or system, examining them for indications of security issues. As system assaults have expanded in number and seriousness in the course of recent years, interruption recognition frameworks have turned into an essential expansion to the security foundation of generally associations. This direction archive is planned as a preliminary in interruption recognition, created for the individuals who need to comprehend what security objectives interruption location components serve, how to choose and design interruption discovery frameworks for their particular framework and system situations, how to deal with the yield of interruption identification frameworks, and how to incorporate interruption recognition capacities with whatever remains of the authoritative security foundation. References to other data sources are likewise accommodated the peruse who requires particular or more point by point guidance on particular interruption identification issues.
In the most recent years there has been an expanding enthusiasm for the security of process control and SCADA frameworks. Moreover, ongoing PC assaults, for example, the Stunt worm, host appeared there are gatherings with the inspiration and assets to viably assault control frameworks.
While past work has proposed new security components for control frameworks, few of them have investigated new and in a general sense distinctive research issues for anchoring control frameworks when contrasted with anchoring conventional data innovation (IT) frameworks. Specifically, the complexity of new malware assaulting control frameworks - malware including zero-days assaults, rootkits made for control frameworks, and programming marked by confided in declaration specialists - has demonstrated that it is exceptionally hard to avert and identify these assaults dependent on IT framework data.
In this paper we demonstrate how, by joining information of the physical framework under control, we can distinguish PC assaults that change the conduct of the focused on control framework. By utilizing information of the physical framework we can center around the last goal of the assault, and not on the specific instruments of how vulnerabilities are misused, and how ...
Courtney Pachucki, IT Specialist at MePush, wrote this amazing Internet hygiene presentation for users on the Web to stay safe and avoid being hacked, phished, or infected with malware. This is a basic set of guidelines to help you identify your risks on the web.
Security and Ethical Challenges Contributors Kim Wanders.docxedgar6wallace88877
Security and Ethical Challenges
Contributors: Kim Wandersee, Les Pang
Computer Security
Computer Security Goals
Computer security must be viewed in a holistic manner and provide an end-to-end protection
as data moves through its lifecycle. Data originates from a user or sensor, passes over a
network to reach a computing system that hosts software. This computer system has software
and processes the data and stores in in a storage device. That data is backed up on a device
and finally archived. The elements that handle the data need to be secure. Computer security
pertains to all the means to protect the confidentiality, integrity, availability, authenticity,
utility, and possession of data throughout its lifecycle.
Confidentiality: A security principle that
works to ensure that data is not disclosed to
unauthorized persons.
Integrity: A security principle that makes sure
that information and systems are not
modified maliciously or accidentally.
Availability: A security principle that assures
reliable and timely access to data and
resources by authorized individuals.
Authenticity: A security principle that the
data, transactions, communications or
documents are genuine, valid, and not
fraudulent.
Utility: A security principle that addresses
that the information is usable for its intended
purpose. .
Possession: A security principle that works to
ensure that data remains under the control of
the authorized individuals.
Figure 1. Parkerian Hexad (PH) security model.
The Parerian Hexad (PH) model expands on the Confidentiality, Integrity, and Availability (CIA)
triad that has been the basic model of Information Security for over 20 years. This framework is
used to list all aspects of security at a basic level. It provides a complete security framework to
provide the means for information owners to protect their information from any adversaries
and vulnerabilities. It adds Authenticity, Utility, and Possession to CIA triad security model. It
addresses security aspects for data throughout its lifecycle.
The Center for Internet Security has identified 20 controls necessary to protect an organization
from known cyber-attack. The first 5 controls will provide effective defense against the most
common cyber-attacks, approximately 85% of attacks. The 5 controls are:
1. Inventory of Authorized and Devices
2. Inventory of Authorized and Unauthorized Software
3. Secure Configurations for Hardware and Software
4. Continuous Vulnerability Assessment and Remediation
5. Controlled User of Administrative Privileges
A full explanation of all 20 controls is available at the Center for Internet Security website.
Search for CIS controls.
Security Standards and Regulations
The National Institute of Standards and Technology (NIST), Computer Security Division, provides
security standards in its Federal Information Processing Standards (.
Security and Ethical Challenges Contributors Kim Wanders.docxfathwaitewalter
Security and Ethical Challenges
Contributors: Kim Wandersee, Les Pang
Computer Security
Computer Security Goals
Computer security must be viewed in a holistic manner and provide an end-to-end protection
as data moves through its lifecycle. Data originates from a user or sensor, passes over a
network to reach a computing system that hosts software. This computer system has software
and processes the data and stores in in a storage device. That data is backed up on a device
and finally archived. The elements that handle the data need to be secure. Computer security
pertains to all the means to protect the confidentiality, integrity, availability, authenticity,
utility, and possession of data throughout its lifecycle.
Confidentiality: A security principle that
works to ensure that data is not disclosed to
unauthorized persons.
Integrity: A security principle that makes sure
that information and systems are not
modified maliciously or accidentally.
Availability: A security principle that assures
reliable and timely access to data and
resources by authorized individuals.
Authenticity: A security principle that the
data, transactions, communications or
documents are genuine, valid, and not
fraudulent.
Utility: A security principle that addresses
that the information is usable for its intended
purpose. .
Possession: A security principle that works to
ensure that data remains under the control of
the authorized individuals.
Figure 1. Parkerian Hexad (PH) security model.
The Parerian Hexad (PH) model expands on the Confidentiality, Integrity, and Availability (CIA)
triad that has been the basic model of Information Security for over 20 years. This framework is
used to list all aspects of security at a basic level. It provides a complete security framework to
provide the means for information owners to protect their information from any adversaries
and vulnerabilities. It adds Authenticity, Utility, and Possession to CIA triad security model. It
addresses security aspects for data throughout its lifecycle.
The Center for Internet Security has identified 20 controls necessary to protect an organization
from known cyber-attack. The first 5 controls will provide effective defense against the most
common cyber-attacks, approximately 85% of attacks. The 5 controls are:
1. Inventory of Authorized and Devices
2. Inventory of Authorized and Unauthorized Software
3. Secure Configurations for Hardware and Software
4. Continuous Vulnerability Assessment and Remediation
5. Controlled User of Administrative Privileges
A full explanation of all 20 controls is available at the Center for Internet Security website.
Search for CIS controls.
Security Standards and Regulations
The National Institute of Standards and Technology (NIST), Computer Security Division, provides
security standards in its Federal Information Processing Standards ( ...
Running Head RISK, THREAT AND VULNERABILITY MANAGEMENT .docxjeanettehully
Running Head: RISK, THREAT AND VULNERABILITY MANAGEMENT 1
RISK, THREAT AND VULNERABILITY MANAGEMENT 2
Risk, Threat and Vulnerability Management
1. Introduction
A general review of threats and vulnerabilities on IT systems was carried out by an American based company known as Para Delta that deals with information technology and sells electronic software. This was done in order to assist the firms to come up with effective security control measures which reduce the risk and threats on the IT networks. Para Delta Company developed procedures through which threat and vulnerability can be managed. The management steps provide emphasis on advance action of network security tasks such as insertion testing. Some automatic systems have advanced antivirus software installed in them, which are not able to identify the specific security threat and vulnerability even though they are capable of detecting dangers. The Para Delta came up with solutions to these threats by creating a threat intelligence foundation that combines human capability and data-driven intersection. Cyber-attacks and risk management are done by first assessing vulnerabilities that help to identify the common threats and the magnitude of their effects on the manufacturing environment. The right set of security arrangements and risk management procedures are required to avoid cybersecurity vulnerabilities that pose serious threats to IT networks. The company found out that there is a need to develop guidelines and techniques which avail adequate information security to secure the operating system. By protecting the information and information systems create an affirmative foundation for strong information. This initiative helps to mitigate risks on the IT networks by protecting it from unauthorized access or destruction. Frameworks given by IT security procedures provide management to the information technology and governance. Frameworks also acknowledge IT governance objectives and good actions by the IT process. Companies are required to develop policies on the planning processes of information security systems, which again require plans of action for implementing security controls. This makes it possible for the provision of a more confidential information system and its availability.
2. The Analysis of Security Baseline
The analysis was carried out by the Para Delta Company through the identification of various procedures, security requirements, the security attacks to the enterprise network control systems, and network infrastructure with security posture components.
2.1. Security requirements and goals
The Para Delta Company carried out an analysis of security baseline through which the identification of various security necessities and results were listed for the preparation of any action of security baseline. The company found out that for strong networks of IT control system to be achieved ...
Running Head RISK, THREAT AND VULNERABILITY MANAGEMENT .docxtodd521
Running Head: RISK, THREAT AND VULNERABILITY MANAGEMENT 1
RISK, THREAT AND VULNERABILITY MANAGEMENT 2
Risk, Threat and Vulnerability Management
1. Introduction
A general review of threats and vulnerabilities on IT systems was carried out by an American based company known as Para Delta that deals with information technology and sells electronic software. This was done in order to assist the firms to come up with effective security control measures which reduce the risk and threats on the IT networks. Para Delta Company developed procedures through which threat and vulnerability can be managed. The management steps provide emphasis on advance action of network security tasks such as insertion testing. Some automatic systems have advanced antivirus software installed in them, which are not able to identify the specific security threat and vulnerability even though they are capable of detecting dangers. The Para Delta came up with solutions to these threats by creating a threat intelligence foundation that combines human capability and data-driven intersection. Cyber-attacks and risk management are done by first assessing vulnerabilities that help to identify the common threats and the magnitude of their effects on the manufacturing environment. The right set of security arrangements and risk management procedures are required to avoid cybersecurity vulnerabilities that pose serious threats to IT networks. The company found out that there is a need to develop guidelines and techniques which avail adequate information security to secure the operating system. By protecting the information and information systems create an affirmative foundation for strong information. This initiative helps to mitigate risks on the IT networks by protecting it from unauthorized access or destruction. Frameworks given by IT security procedures provide management to the information technology and governance. Frameworks also acknowledge IT governance objectives and good actions by the IT process. Companies are required to develop policies on the planning processes of information security systems, which again require plans of action for implementing security controls. This makes it possible for the provision of a more confidential information system and its availability.
2. The Analysis of Security Baseline
The analysis was carried out by the Para Delta Company through the identification of various procedures, security requirements, the security attacks to the enterprise network control systems, and network infrastructure with security posture components.
2.1. Security requirements and goals
The Para Delta Company carried out an analysis of security baseline through which the identification of various security necessities and results were listed for the preparation of any action of security baseline. The company found out that for strong networks of IT control system to be achieved.
INDUSTRY-LEADING TECHNOLOGY FOR LONG TERM RETENTION OF BACKUPS IN THE CLOUDEMC
CloudBoost is a cloud-enabling solution from EMC
Facilitates secure, automatic, efficient data transfer to private and public clouds for Long-Term Retention (LTR) of backups. Seamlessly extends existing data protection solutions to elastic, resilient, scale-out cloud storage
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIOEMC
With EMC XtremIO all-flash array, improve
1) your competitive agility with real-time analytics & development
2) your infrastructure agility with elastic provisioning for performance & capacity
3) your TCO with 50% lower capex and opex and double the storage lifecycle.
• Citrix & EMC XtremIO: Better Together
• XtremIO Design Fundamentals for VDI
• Citrix XenDesktop & XtremIO
-- Image Management & Storage
-- Demonstrations
-- XtremIO XenDesktop Integration
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES EMC
Explore findings from the EMC Forum IT Study and learn how cloud computing, social, mobile, and big data megatrends are shaping IT as a business driver globally.
Reference architecture with MIRANTIS OPENSTACK PLATFORM.The changes that are going on in IT with disruptions from technology, business and culture and so IT to solve the issues has to change from moving from traditional models to broker provider model.
Force Cyber Criminals to Shop Elsewhere
Learn the value of having an Identity Management and Governance solution and how retailers today are benefiting by strengthening their defenses and bolstering their Identity Management capabilities.
Container-based technology has experienced a recent revival and is becoming adopted at an explosive rate. For those that are new to the conversation, containers offer a way to virtualize an operating system. This virtualization isolates processes, providing limited visibility and resource utilization to each, such that the processes appear to be running on separate machines. In short, allowing more applications to run on a single machine. Here is a brief timeline of key moments in container history.
This white paper provides an overview of EMC's data protection solutions for the data lake - an active repository to manage varied and complex Big Data workloads
This infographic highlights key stats and messages from the analyst report from J.Gold Associates that addresses the growing economic impact of mobile cybercrime and fraud.
This white paper describes how an intelligence-driven governance, risk management, and compliance (GRC) model can create an efficient, collaborative enterprise GRC strategy across IT, Finance, Operations, and Legal areas.
The Trust Paradox: Access Management and Trust in an Insecure AgeEMC
This white paper discusses the results of a CIO UK survey on a“Trust Paradox,” defined as employees and business partners being both the weakest link in an organization’s security as well as trusted agents in achieving the company’s goals.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Epistemic Interaction - tuning interfaces to provide information for AI support
Beehive: Large-Scale Log Analysis for Detecting Suspicious Activity in Enterprise Networks
1. Beehive: Large-Scale Log Analysis for Detecting
Suspicious Activity in Enterprise Networks
Ting-Fang Yen
Alina Oprea
Kaan Onarlioglu
RSA Laboratories
Cambridge, MA, USA
RSA Laboratories
Cambridge, MA, USA
Northeastern University
Boston, MA, USA
onarliog@ccs.neu.edu
tingfang.yen@rsa.com
aoprea@rsa.com
Todd Leetham
William Robertson
EMC Corp
Hopkinton, MA, USA
Northeastern University
Boston, MA, USA
todd.leetham@emc.com
Ari Juels
wkr@ccs.neu.edu
RSA Laboratories
Cambridge, MA, USA
Northeastern University
Boston, MA, USA
Engin Kirda
ek@ccs.neu.edu
ajuels@rsa.com
ABSTRACT
1.
As more and more Internet-based attacks arise, organizations are responding by deploying an assortment of security
products that generate situational intelligence in the form
of logs. These logs often contain high volumes of interesting and useful information about activities in the network, and are among the first data sources that information security specialists consult when they suspect that an
attack has taken place. However, security products often
come from a patchwork of vendors, and are inconsistently
installed and administered. They generate logs whose formats differ widely and that are often incomplete, mutually
contradictory, and very large in volume. Hence, although
this collected information is useful, it is often dirty.
We present a novel system, Beehive, that attacks the problem of automatically mining and extracting knowledge from
the dirty log data produced by a wide variety of security
products in a large enterprise. We improve on signaturebased approaches to detecting security incidents and instead
identify suspicious host behaviors that Beehive reports as
potential security incidents. These incidents can then be
further analyzed by incident response teams to determine
whether a policy violation or attack has occurred. We have
evaluated Beehive on the log data collected in a large enterprise, EMC, over a period of two weeks. We compare the
incidents identified by Beehive against enterprise Security
Operations Center reports, antivirus software alerts, and
feedback from enterprise security specialists. We show that
Beehive is able to identify malicious events and policy violations which would otherwise go undetected.
Protection of computing infrastructure in large organizations is a mounting challenge. Both generic malware and
targeted attacks (i.e., Advanced Persistent Threats (APTs))
are growing in sophistication and outstripping the capabilities of traditional defenses such as antivirus software. At
the same time, administrators’ visibility into the posture
and behavior of hosts is eroding as employees increasingly
use personal devices for business applications. The traditional perimeters of organizations are breaking down, moreover, due to new complexities in business intelligence sharing, contractor relationships, and geographical distribution.
Many organizations are responding to this challenging landscape by deploying an assortment of security products that
enforce policies and generate situational intelligence in the
form of security logs. These products proxy web traffic,
detect malware infections, check for policy violations, implement effective authentication mechanisms, and perform
many other useful security functions.
These products yield logs that contain high volumes of
interesting and useful information about activities in the
network, and are among the first data sources consulted by
security specialists when an attack is in evidence. For example, authentication logs might suggest that an employee’s
account has been compromised because a login has occurred
from a region that the targeted employee has never visited.
As another example, web proxy logs might record which
site a victim visited before being compromised by a driveby download attack.
While it is standard practice to examine logs to discover or
conduct forensic analysis of suspicious activity in a network,
such investigation remains a largely manual process and often relies on signatures of known threats. A major challenge
is that security products often come from a patchwork of
vendors and are inconsistently installed and administered.
They produce log data with widely differing formats, and
logs that are often incomplete, mutually contradictory, very
large in volume (e.g., security-relevant logs in a large enterprise grow by terabytes per day), and filled with non-specific
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
ACSAC ’13 Dec. 9-13, 2013, New Orleans, Louisiana, USA
Copyright 2013 ACM 978-1-4503-2015-3/13/12 ...$15.00.
http://dx.doi.org/10.1145/2523649.2523670
INTRODUCTION
2. or spurious event records. Hence, although this collected information is useful, and is sometimes the only information
at hand, at the same time it is often dirty.
In this paper, we attack the problem of automatically
mining and extracting knowledge from the dirty log data
produced by a wide variety of security products in a large
enterprise. Our aim is to improve on signature-based approaches to detecting security incidents and instead achieve
behavioral detection of suspicious host activities that signal potential security incidents. These incidents can then
be further analyzed by incident response teams to determine whether a policy violation or attack has occurred. A
key insight of our approach is that behaviors observed in
enterprise settings are often significantly more constrained
by policy and typical employee behavior than those on the
open Internet. Thus we can effectively identify suspicious
activities by analyzing behaviors in enterprise-specific ways.
Our automated approach, called Beehive, has three layers:
(1) Functionality to parse, filter, and normalize log data using network-specific configuration information; (2) Featuregenerators that process normalized log data to produce a set
of fifteen distinct features for each host per day; and (3) A
detector that performs clustering over features to identify
outlying, suspicious periods of host activity, which the system reports as incidents.
We have evaluated Beehive on the log data collected in
a large enterprise, EMC, over a period of two weeks. We
compare the incidents automatically identified by Beehive
against enterprise Security Operations Center reports, antivirus software alerts, and feedback received from information security specialists in the company. We show that Beehive is able to identify a number of malicious events within
the enterprise network, including malware infections and
policy violations, that otherwise go unnoticed by existing,
state-of-the-art security tools and personnel.
Our work makes the following contributions:
• We propose a novel automated approach called Beehive that analyzes very large volumes of disparate log
data collected in large organizations to detect malicious activity such as malware infections and policy
violations. Beehive leverages unique features of enterprise networks to produce accurate, actionable information.
• We have evaluated Beehive on more than 6 terabytes of
log data and have correctly identified suspicious activities that went unnoticed by existing state-of-the-art
security tools and personnel.
• To the best of our knowledge, ours is the first exploration of the challenges of “big data” security analytics
for large volumes of disparate, real-world log data.
2.
PROBLEM AND CHALLENGES
Here we describe the threat model we address, the nature of the log data on which the system operates, and the
challenges that arise in designing and evaluating Beehive.
2.1
Threat Model
Beehive aims to detect threats arising in two different scenarios. The first scenario involves a network host that is
infected or at imminent risk of infection. A host of this
kind may exhibit malicious or dangerous behaviors such as
contacting an attack website, communicating with a C&C
server, or exfiltrating data to a previously unseen external
destination. These behaviors may result from the automated
activities of malware (e.g, zombie machines in a botnet), direct control of hosts by an attacker (e.g., through a remote
access Trojan), or benign users being tricked into dangerous
behavior (e.g., via phishing attacks).
The second scenario concerns host-based user activities
that violate enterprise policies. These activities are not necessarily malicious in intent, but are potentially damaging to
the business, and thus undesirable. Examples include peerto-peer file sharing, instant messaging, multimedia streaming, viewing of adult content, online gaming, and using proxies and tunneling services in an attempt to bypass enterprise
firewalls and other network-perimeter security mechanisms.
2.2
Data
Beehive uses a wide range of logs generated by various network devices, including web proxies that log every outbound
connection, DHCP servers that log dynamic IP address assignments, VPN servers that log remote connections to the
enterprise network, Windows domain controllers that log authentication attempts within the corporate domain, and antivirus software that logs the results of malware scans on
end hosts. These logs are stored in a commercial security
information and event management (SIEM) system.
All of these devices monitor and log the activities of enterprise hosts transparently, with the notable exception of the
web proxy. For every external destination, the web proxy
consults a list of domain reputation scores and site categories
maintained by the product vendor, and automatically blocks
the connection if the destination has a low reputation or is in
a prohibited category (e.g., adult-content sites). However,
if the destination is not on this list, the proxy displays to
the user a warning page describing the potential dangers of
connecting to an unknown destination. The user must then
explicitly click on a link to acknowledge agreement to the
company’s network policies, or not visit the site.
Beehive makes heavy use of the web proxy logs, since the
majority of network activity in the enterprise is over HTTP
or HTTPS. In addition, these logs include all fields in HTTP
headers, making them a particularly useful resource for understanding users’ browsing behavior, detecting suspicious
connections, and forensic analysis. Table 1 lists a subset of
the fields included in web proxy logs.
IP Header
Source/destination IP and port,
protocol
Transport Header
Destination domain,
URL,
HTTP status code, web referer,
domain reputation,
domain
category, user-agent string
Connection Attribute
Sent and received bytes, network
policy
Table 1: Fields in web proxy logs.
At EMC, 1.4 billion log messages are generated daily on
average, at a rate of around one terabyte a day. These “raw”
logs contain huge amounts of information about user and
host behaviors. They are also noisy — with non-standardized
timestamps, different identifiers (i.e., some store a host’s IP
3. address, some the hostname, others the username) — and
can be truncated or arrive out of order. We discuss the
challenges of analyzing such “big data” in the next section.
2.3
Challenges
The massive number of events observed on a real-life enterprise network and logged inside the SIEM system poses a
major challenge (a “big data” problem) in log analysis and
detection of security incidents. As noted above, we observed
collection by the SIEM an average of 1.4 billion logs per
day in our case study. Timely detection of critical threats
by Beehive thus requires efficient data-reduction algorithms
and strategies to focus on security-relevant information in
the logs.
A second challenge for Beehive is that of identifying meaningful security incidents in the face of a significant semantic
gap between the logs collected by the SIEM system and the
information that security analysts require to identify suspicious host behavior. In our case, Beehive needs to produce
a daily report of suspicious hosts; however, most network
devices only log the IP addresses of network endpoints. Associating IP addresses with specific hosts requires correlation across different logs, which can be especially challenging
when IP addresses are dynamically assigned in the network.
The situation is complicated still more by the fact that the
logging devices may be located in different time zones (as is
typical for the infrastructure of a global organization), and
thus the timestamps they produce must be normalized for
accurate temporal correlation.
Finally, a major challenge in designing Beehive is the difficulty of evaluating its effectiveness accurately, due to an
inherent lack of ground truth. Specifically, an enterprise
network typically has state-of-the-art security technologies
deployed both on the network perimeter and on hosts, and
security analysts actively work toward resolving known incidents. As a result, real-life network data obtained from
such a network often lacks traces of known security threats.
For example, malware on hosts is often cleaned or quarantined before becoming active, and accesses to botnet C&C
servers are often blocked by the firewall. Such threat resolution prevents automatic testing of Beehive over a ground
truth of known malicious network traffic, as none is in evidence. Many of the incidents identified by Beehive are in
fact unknown to security systems in use in the enterprise.
We therefore must rely on manual evaluation of the security
incidents and suspicious hosts identified by Beehive.
In designing Beehive, we explore and tackle these challenges of big data security analytics. Briefly, in a pre-processing
phase, Beehive normalizes the timestamps of all log entries
to UTC, determines statically and dynamically assigned IP
addresses of hosts, and constructs bindings between hosts
and IP addresses, thus attributing every logged event to a
specific host. Beehive then extracts enterprise-specific features for individual hosts, and clusters hosts with similar
suspicious behaviors. The resulting outlying clusters are
presented as incidents to be examined by security analysts.
To assess the efficacy of Beehive in this study, we manually investigate its reported incidents with the help of our
enterprise’s Security Operations Center (SOC). Given existing enterprise resolution of attacks with known signatures,
of particular interest in our study is Beehive’s identification
of network policy violations by users and of malware not
detected by current signature-based tools.
3.
THE BEEHIVE SYSTEM
In this section we describe in detail the operation of the
Beehive system. We start by describing our approach to
remove noise and inconsistencies in the data collected by the
SIEM. We then discuss the selection of features distinctive to
an enterprise setting for detecting misbehavior. Finally, we
describe the unsupervised learning approach we employed
for generating incidents of anomalous host behavior.
3.1
Data Normalization
In order to address the challenges of dirty and inconsistent data explained in the previous sections, Beehive preprocesses the log data before starting its behavioral-analysis.
We explain different stages of this process below.
Timestamp Normalization. In an enterprise running a
global network, devices that produce critical logs are located
in different geographies. Moreover, these devices may timestamp logs differently, using their local time zones, UTC, or
other time representations. This results in log entries with
inconsistent timestamps that must be normalized to a standard representation before any temporal analysis becomes
accurate. While delegating the coordination of time settings to network administrators may be feasible for small
networks, these approaches are unworkable at the scale of a
global enterprise.
Beehive addresses this problem by leveraging the common
use by enterprises of a central SIEM system for log management that tags each log entry with its own timestamp
tsiem , recording the time at which the log was received by
the SIEM. For each device that sends logs to the SIEM system, we first compute a set of time difference values ∆i =
tsiem,i −tdevice,i (rounded off to the nearest 30 minutes) from
each log entry i generated over a large time period (e.g., one
month). Next, we determine the timestamp correction value
∆correction for the device by setting it to the value ∆i that
accounts for the majority of differences. Applying this correction value to each device timestamp gives us a normalized
timestamp value, tnormalized,i = tdevice,i + ∆correction . In
our case, as EMC’s SIEM system is configured to use UTC
timestamps, tnormalized,i corresponds to UTC time. We apply this normalization technique to each device on the network that produces log data, normalizing all log timestamps
to UTC.
IP address-to-Host Mapping. In an enterprise setting,
end hosts are usually assigned IP addresses dynamically
for a short time period when they connect to the network,
through the Dynamic Host Configuration Protocol (DHCP).
This makes it difficult to associate the IP addresses reported
in logs with unique host machines during analysis, because
that same IP address could be assigned to different hosts
after the event has been logged.
To address this issue, Beehive analyzes the DHCP server
logs collected in the SIEM system and constructs a database
of IP-to-host mappings (i.e., bindings) over time. Each binding is represented as a tuple {IP address, hostname, MAC
address, start-time, end-time} mapping an IP address to a
host in a specific time interval. This algorithm is run daily
to update existing bindings as new DHCP logs become available. Given the resulting database of bindings, Beehive can
identify the host corresponding to a given IP address with
reference to a normalized timestamp.
4. Static IP Address Detection. An IP-to-host lookup on
the bindings database we construct may fail because an IP
address is assigned statically to a specific host, not dynamically leased. Maintaining a list of static IP address assignments, however, is difficult in a large enterprise network,
and administrator-created lists are often non-existant or outdated. Thus Beehive also adopts the following method for
automatically identifying hosts with static IP addresses.
In the bootstrap step, we first retrieve all IP addresses
found in all collected logs to create a pool of IP addresses
active in the enterprise, denoted by set A. Next, we retrieve
IP addresses from logs that we know must only contain hosts
with dynamic IP addresses, such as DHCP and VPN logs,
to create a pool of known dynamic IP addresses, denoted by
set D. We then compute the set difference S = A − D which
contains potentially static IP addresses, perform a reverse
DNS lookup for every address in S, and save the results
to complete the bootstrap phase. Periodically (e.g., once a
day), as new logs become available, we repeat this procedure
to harvest new IP addresses and update the sets A, D and S
accordingly. However, unlike before, with each iteration we
resolve IP addresses in S to their host names and compare
the names to the previously stored values. If the host names
changed between two iterations, we conclude that the given
IP address is not statically assigned, and we remove it from
the set S. In this way, we refine the pool of potentially static
IP addresses with each iteration. If Beehive fails to find a
corresponding binding for an IP-to-host lookup, but instead
finds the given address in S, we treat that IP address as a
static host and use the host name found in S.
Dedicated Hosts. Beehive focuses on monitoring the behavior of dedicated hosts, which are machines utilized by a
single user. Since a list of dedicated hosts is difficult to maintain in large enterprises due to constantly changing network
configurations, we infer the list of dedicated hosts from the
data available in the SIEM system.
We make use of authentication logs generated by Microsoft Windows domain controllers for this purpose. For
each user in the enterprise, a history is kept of hosts onto
which the user had authenticated (i.e., logged on), the number of authentications, and the authentication timestamps.
This information is collected over the course of three months
to build an accurate history of user activity. At the end of
the collection period, we consider a host as “dedicated” if a
single user is responsible for the large majority (e.g., 95%)
of the authentication events on that host. Through this process, we have identified over 78,000 dedicated hosts at EMC.
3.2
Feature Extraction
We extracted features from the logs to characterize outbound communications from the enterprise. Our feature selection is guided by observation of known malware behaviors and policy violations within EMC, as well as properties of the environment in which Beehive operates, i.e., the
presence of perimeter defenses in the form of strict firewall
policies, the business orientation of (most) users’ activities,
and the (relatively) homogeneous software configurations of
enterprise-managed hosts.
For each dedicated host in the enterprise, we generate
daily a feature vector that includes the 15 features listed in
Table 2. The features can be grouped into four categories:
features based on new and unpopular destinations contacted
by the host, features related to the host’s software configu-
ration, features related to the company policy, and features
based on traffic volume and timing. We describe each of
them in detail below.
Feature Type
#
Description
DestinationBased
1
2
3
4
New destinations
New dests. w/o whitelisted referer
Unpopular raw IP destinations
Fraction of unpopular raw IP dests.
Host-Based
5
New user-agent strings
PolicyBased
6
7
8
9
10
11
Blocked domains
Blocked connections
Challenged domains
Challenged connections
Consented domains
Consented connections
TrafficBased
12
13
14
15
Connection spikes
Domain spikes
Connections bursts
Domain bursts
Table 2: Beehive features.
3.2.1
Destination-Based Features
We are interested in identifying hosts that communicate
with new, or obscure, external destinations that are never
(or rarely) contacted from within EMC. Assuming popular
websites are better administered and less likely to be compromised, connections to uncommon destinations may be
indicative of suspicious behavior (e.g., communication with
command-and-control servers).
New Destinations. Our first destination-based feature is
the number of new external destinations contacted by each
host per day. We build a history of external destinations
contacted by internal hosts over time. After an initial bootstrapping period of one month, we consider a destination to
be new on a particular day if it has never been contacted by
hosts in the enterprise within the observation period (and
as such is not part of the history). On a daily basis we update the history to include new destinations contacted in the
previous day.
We encountered a number of challenges when analyzing
web proxy logs to build the history of external destinations.
In our initial, na¨ implementation, we process every proxy
ıve,
log, resolve all destinations that are IP addresses, and include all observed new destinations in the history. It turns
out that this na¨ approach is not scalable both in terms of
ıve
running time and history size.
First, the na¨ implementation takes 15 hours to process
ıve
a day’s worth of web proxy logs (the equivalent of 300 million
logs, or 600 GB). Second, to our surprise, the number of new
destinations does not decrease over time, even as the size of
the history grows to 4.3 million unique destinations over the
course of a month. As shown in Figure 1, between 30% to
40% (about 145,000) of all unique destinations each day are
new. In the na¨ approach, all of these new destinations
ıve
are added to the history daily, and as such the size of the
history grows indefinitely over time.
Further inspection of the web proxy logs showed that the
majority of the new destinations are content delivery net-
5. works (CDNs), cloud services (which frequently use random
strings as subdomains), or IP addresses belonging to popular services (Google, Facebook, Twitter). To make Beehive
scalable, we employ a number of data reduction techniques,
including filtering, custom whitelisting and domain “folding.”
Unique destinations per day
Unique new destinations per day
Destinations in history (naive approach)
Number of unique destinations
5e+06
New destinations without whitelisted referer. Our
second destination-based feature is an extension of the first
one, but counts the number of “new” destinations contacted
by a host without a whitelisted HTTP referer. Users are
most commonly directed to new sites by search engines, news
sites, or advertisements. In these cases, we expect the HTTP
referer to be one listed in our custom whitelist. Host that
visit new sites without being referred by a reputable source
(or with no referer at all) are considered more suspicious.
4e+06
3e+06
2e+06
1e+06
0
0
5
10
15
Day
20
25
30
Figure 1: Destinations contacted by internal hosts,
na¨ approach.
ıve
We first filter “popular” destinations by creating a custom whitelist, where “popularity” is defined over hosts in
the enterprise. The whitelist includes external destinations
(both domains and IP subnets) whose number of interacting internal hosts over time (i.e., a training period of one
week) exceeds a threshold. Figure 2 shows the amount of
web traffic filtered using a whitelist constructed from the
first week’s worth of data from April 2013. A threshold of
100 hosts achieves a reduction from 300 million logs a day to
80 million—a 74% reduction in the number of logs Beehive
needs to process.
All logs
Doms < 1000 hosts
Doms < 500 hosts
Doms < 200 hosts
Doms < 100 hosts
4e+08
Number of logs
3.5e+08
3e+08
2.5e+08
2e+08
1.5e+08
1e+08
5e+07
0
Mon
Tue
Wed
Thu
Day
Fri
choose not to resolve raw IPs (since most legitimate sites are
referred to by their domain name), and always consider raw
IPs that are not on the whitelist as “new”.
These optimizations reduce daily processing time from 15
hours to about five hours. The number of new (folded) destinations added to the history is reduced to an average of
28,000 per day. After a period of four months (from January
13th to May 13th, 2013), our history of external destinations
has 2.7 million folded domains, whereas the history built
na¨
ıvely already had 4.3 million domains after one month.
Sat
Sun
Figure 2: Data reduction through custom whitelisting.
In addition to custom whitelisting, we “fold” destinations
to the second-level domain so as to filter services employing
random strings as subdomains. We also ignore connections
retrieving “favicon” — likely due to bookmarks. Finally, we
Unpopular IP destinations. We are also interested in
unpopular external destinations that are raw IP addresses.
We first count the number of destinations contacted by a
host that are both unpopular (not on the custom whitelist
described above) and are IP addresses. Connections to unpopular IPs can indicate suspicious activity, as legitimate
services can usually be reached by their domain names. Second, we include the fraction of unpopular destinations contacted by a host that day that are IP addresses. While occasional communication with IP destinations is normal (e.g.,
to IP ranges owned by CDNs), such behavior is suspicious
when frequent.
3.2.2
Host-Based Features
Hosts in an enterprise are significantly more homogeneous
in their software configurations than, for example, those in
academic networks. Hence we are interested in cases where
hosts install new (and potentially unauthorized) software.
Lacking visibility onto the host machine, and having access only to logs collected from network devices, we infer
the software configurations on a host from the user-agent
(UA) strings included in HTTP request headers. A useragent string includes the name of the application making
the request, its version, capabilities, and the operating environment. Our host-based feature is hence the number of
“new” UA strings from the host.
We build a history of UA strings per host over a monthlong period, during which every UA string observed from the
host is stored. Afterwards, a UA string is considered “new” if
it is sufficiently distinct from all UA strings in that host’s history. Edit distance (also called Levenshtein distance) is used
to compare UA strings, measuring the number of character
insertions, deletions, and substitutions required to change
one string into another. This allows us to accommodate
“new” UA strings that result from software updates, where
only a small substring (e.g., the version number) changes.
3.2.3
Policy-Based Features
Also unique to enterprise environments is the enforcement
of network policies on outbound connections. As described
in Section 2.2, a connection to an external destination can
be blocked if it has a dubious reputation or is categorized
as prohibited for employees. Blocked domains (and connec-
6. tions) are thus a coarse indicator of host misbehavior.
Upon visiting an unknown destination, i.e., one that has
not yet been categorized or rated, the user must explicitly
agree to adhere to the company’s policies before being allowed to proceed. We refer to the domains (and connections)
that require this acknowledgment as challenged, and those
to which the user has agreed as consented.
Our policy-based features include all three types of communications described above. For a host, we count the number of domains (and connections) contacted by the host that
are blocked, challenged, or consented.
3.2.4
Traffic-Based Features
Sudden spikes in a host’s traffic volume can be caused
by malware (e.g., scanning, or bot activities in response to
botmaster commands) or automated processes. Our trafficbased features attempt to capture these interesting activities
by the amount of time when a host is generating abnormally
high volumes of traffic.
Specifically, we define a connection (or domain) spike as
a one-minute window when the host generates more connections (or contacts more domains) than a threshold. A
connection (or domain) burst is a time interval in which every minute is a connection (or domain) spike.
To identify an appropriate threshold for “high” traffic volume, we examine all dedicated hosts over a one-week interval, and count the number of connections (and domains)
each host generates (or contacts) per minute. Figure 3 shows
the cumulative distribution across all one-minute windows
for all dedicated hosts. 90% of the hosts generated less
than 101 connections, and contacted less than 17 distinct
domains, per minute. We hence set the threshold for connection spikes and domain spikes to these values.
3.3
Clustering
Lacking ground truth as to which hosts are infected or behaving anomalously (a challenge we describe in Section 2.3),
we tackle the problem of detection through an unsupervised
learning approach – clustering. Since employees in the enterprise perform specific job functions on the corporate network, and there are multiple employees in most departments,
we should be able to observe groups of hosts (belonging to
users with similar roles) exhibiting similar behaviors, while
misbehaving hosts with unique behavioral patterns appear
as outliers.
Given the 15 features described in Section 3.2, each internal host is represented as a 15-dimensional vector, v = (v[1],
v[2], · · · , v[15]). However, the features may be related or dependent upon one another; e.g., a domain spike also triggers
a connection spike. To remove such dependencies between
the features and reduce the dimensionality of the vectors,
we apply Principal Component Analysis (PCA) [23].
Briefly, PCA enables data reduction by projecting the
original vectors onto a new set of axes (i.e., the principal
components). Each principal component is chosen to capture as much of the variance (and thus the original information) in the data as possible. Depending on how much
variance we want to capture from the original data, the top
m principal components are selected, permitting projection
of the original vectors down to dimensionality m. In Beehive, we select the top m components that capture at least
95% of the data variance.
We apply a clustering algorithm to the projected vectors
after PCA. Our algorithm is an adaptation of the K-means
clustering algorithm, but does not require the number of
clusters to be specified in advance [25].
1. Randomly select a vector as the first cluster hub. Assign all vectors to this cluster.
2. Select the vector furthest away from its hub as a new
hub. Reassign every vector to the cluster with the
closest hub.
3. Repeat step 2 until no vector is further away from its
hub than half of the average hub-to-hub distance.
Figure 3: CDF for number of web connections and
number of domains contacted by a host per oneminute interval.
For bursts, we relax its definition slightly, so that the
threshold for spikes within a burst is the 75% percentile of
all one-minute windows across all hosts (see Figure 3). This
value is 26 for connection spikes, and 6 for domain spikes.
For each dedicated host, its traffic-based features include:
1) the number of connections spikes, 2) the number of domain spikes, 3) the duration of the longest connection burst,
and 4) the duration of the longest domain burst.
We compare vectors via L1 distance, i.e., for vectors v1 and
v2 , their distance is L1Dist(v1 , v2 ) = m |v1 [i] − v2 [i]|.
i=1
The clustering algorithm in Beehive is applied daily on the
feature vectors for all active, dedicated hosts in the enterprise. (We observe between 27,000 and 35,000 hosts active
during weekdays, and between 9,000 and 10,100 hosts active on weekends.) Interestingly, after one iteration of the
algorithm, the vast majority of hosts fall into one large cluster, while the remaining clusters consist of few hosts whose
behaviors deviate significantly from the norm (i.e., outliers).
Beehive generates incidents for the top outlying hosts, and
reports them to the security analyst. Note that the algorithm forms clusters by iteratively identifying the node that
is furthest away from its cluster hub, and so the clusters
have an inherent ordering to them. In cases where the algorithm is biased by extreme outliers (e.g., forming only two
clusters, one with a single node, and the other with all other
nodes), we apply PCA and the clustering algorithm again
to the largest cluster. This process is iterated until at least
50 outlying hosts are identified in a day.
7. 4.
EVALUATION
We evaluated Beehive in a large enterprise, EMC, over a
period of two weeks, from April 22 to May 5, 2013. During
this period, Beehive generated 784 incidents, with an average
of 56 incidents per day and a standard deviation of 6.88.
Among these, only eight incidents overlap with the alerts
generated by the enterprise’s state-of-the-art security tools.
To evaluate Beehive in the absence of ground truth – in
the sense of accurate, existing classification of identified incidents – we resort to a two-step process of manual investigation. We first examine and label all incidents generated
by Beehive (see Section 4.2). Incidents whose cause we could
not identify are labeled as “suspicious” and presented to the
enterprise’s Security Operations Center (SOC) for further
analysis (see Section 4.3).
4.1
and when they were registered (using DomainTools 2 ).
Briefly, 25.25% of the Beehive incidents were either confirmed malware or “suspicious” (we give more details on “suspicious” incidents in Section 4.3), 39.41% were violations of
enterprise security policies, and 35.33% were associated with
unrecognized (but noisy) software or services. Table 4 lists
the number and percentage of incidents in each category.
Category
Malware
Suspicious
117
81
14.92%
10.33%
1
2
86
56
6
13
4
8
133
0.12%
0.25%
10.96%
7.14%
0.76%
1.65%
0.51%
1.02%
16.96%
57
63
157
7.27%
8.03%
20.02%
Policy
Policy
Policy
Policy
Policy
Policy
Policy
Policy
Policy
Clustering Context
In addition to the host and its feature vector, each Beehive incident also includes contextual information about the
cluster each host belongs to, such as the number of hosts
in the cluster and the average value of the feature vectors.
This information facilitates the manual investigation process
described below, as it allows the analyst to quickly identify
the distinctive features in each cluster.
To give intuition about the types of activities captured
by the clustering, Figure 4 shows, for all 15 outlying clusters generated on April 24th, the normalized average feature
vectors. That is, we first compute the mean feature vector
of each cluster averaged over all vectors in that cluster, and
then normalize each vector component — i.e., feature value
– relative to the maximum across all mean feature vectors.
As shown in Figure 4, the clusters are distinct in that they
are characterized by different (subsets of) features.
While clusters are distinct from one another, the hosts
in the same cluster exhibit very similar behaviors. Table 3
shows the actual feature vectors of hosts in two example
clusters (3 and 6) on April 24th. For instance, cluster 3 is
unique in its high numbers of blocked connections, connections spikes, and domain spikes — suggesting an automated
process is responsible for the activity. (Indeed, during our
investigation described in the next section, hosts in this cluster are found to violate the company policy by contacting
blocked domains excessively.) As another example, hosts
in cluster 6 contacted high numbers of new destinations, as
well as generating connection spikes to blocked and challenged domains. Our investigation revealed that this cluster
is comprised solely of infected hosts that contact dynamically generated domains (DGAs).
In the following, we detail our manual investigation process and the suspicious activities detected by Beehive.
# of incidents
Violation
Violation
Violation
Violation
Violation
Violation
Violation
Violation
Violation
-
Tunneling
File sharing
Streaming
IM
Porn
Gaming
Proxy
Remote access
Blocked sites
Other - Uncategorized sites
Other - Browsing
Other - Automated
Table 4: Breakdown of the categories of Beehive incidents over two weeks from April 22 to May 5, 2013.
Our manual investigation process for a host began by identifying the feature(s) that were distinctive for the host’s cluster. We then attempted to identify the root cause of this
distinction using contextual information reported by Beehive
and examining raw logs collected by the SIEM system about
the host. The latter involved looking up the UA strings,
HTTP status codes, web referers, and timing patterns in
connections from the host, as well as the reputation of contacted domains (using McAfee SiteAdvisor and URLvoid 1 )
Many of the malware detected by Beehive contacted (or attempted to contact) domains created by domain-generation
algorithms (DGAs). The destination-based features (see
Section 3.2) proved to be most useful in identifying such
malware, as most of the DGA hosts sit in clusters with high
numbers of new destinations. Among clusters from April
24th presented in Figure 4, all of the hosts in clusters 1, 6
and 8 were confirmed to be infected with DGA malware.
None of these infections were previously detected by antivirus software or by the enterprise SOC.
Beehive also detected violations of the enterprise’s network
security policies. As with many companies, these forbid substantive personal use of company computing resources, installation of software not explicitly approved by the company, access to offensive materials (e.g. pornographic or
otherwise disparaging content), and injudiciously high consumption of bandwidth for business purposes. Our manual
investigation found 16.96% of the incidents associated with
heavy traffic to external destinations blocked due to business
concerns or labeled as potentially unsafe (e.g., sites associated with freeware). 10.96% of incidents were caused by
high-volume video streaming applications, often characterized by connection or domain spikes or numerous challenged
connections. Others included file sharing, third-party instant messaging, online gaming, viewing of adult content, or
using tunneling or proxy services to bypass network security
mechanisms (e.g., to access blocked websites).
In addition to malware and policy violations, Beehive also
detected unusual activities originating from possibly benign,
but unrecognized and likely automated applications. For
example, 20.02% of the incidents involved hosts that made
tens of thousands of connections a day to the same website.
These were typically news, sports, or finance sites, but included ad servers, online forums, and sites hosted on Google
1
2
4.2
Manual Labeling
http://www.urlvoid.com
http://www.domaintools.com
8. Figure 4: Normalized average feature vectors for clusters generated on April 24, 2013.
Features
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Cluster 3
0
0
0
0
19
2
0.95
0.33
0
0
4
11
47833
25479
7
1
52
1
0
0
0
0
386
309
96
6
2
2
2
1
Cluster 6
247
200
239
214
247
200
239
214
11
14
22
0
0.041
0.053
0.080
0
1
0
0
0
8
36
26
31
61
435
976
739
156
142
153
142
163
170
177
147
0
6
0
0
0
10
0
0
11
27
26
0
19
34
29
0
1
2
2
0
1
2
1
0
Maximum value
490
489
286
0.984
4
114
47833
418
16373
64
3832
386
213
48
48
Table 3: Feature vectors for hosts in two example clusters. Each row corresponds to the feature vector for a
host in that cluster. The last row shows the maximum value for each feature across all clusters on April 24,
2013.
App Engine. Several hosts were found to be mining bitcoins.
We labeled all of these hosts as “Other - Automated.”
Finally, 8.03% of the incidents either had high numbers
(thousands per day) of consented connections, indicating
that the user had explicitly acknowledged the company’s
network policies before proceeding to the site, or long periods (i.e., lasting several hours) of continuous web traffic
to various domains. Without further evidence of malicious
behavior, we classified these incidents in the “Other - Browsing” category, though they may not be entirely user-driven.
4.3
SOC label
# of suspicious incidents
Adware or Spyware
Further investigation
Other malware
Policy Violation - Gaming
Policy Violation - IM
Policy Violation - Streaming
Other - Uncategorized sites
35
26
9
1
1
2
7
43.21%
32.09%
11.11%
1.23%
1.23%
2.47%
8.64%
Table 5: “Suspicious” incidents categorized by the
SOC at EMC.
Investigation by the SOC
The first-round manual labeling process described in Section 4.2 yielded 81 incidents (10.33%) for which we could
not identify the applications responsible for the observed
behavior in the incident. The hosts in those incidents communicated with low-reputation websites (e.g., flagged by antivirus vendors, registered within the past six months, with
DGA-like subdomains), downloaded executables or zipped
files, used malformed HTTP user-agent strings, or repeatedly contacted the same URL at semi-regular intervals.
We labeled these incidents as “suspicious,” and presented
them to the enterprise SOC for a second round of analysis.
Table 5 shows the labels the SOC assigned to these incidents.
While 54.32% of the “suspicious” incidents were confirmed
by the SOC as adware, spyware, or known malware, a significant fraction (32.09%) were considered serious incidents and
merit further investigation. These questionable activities
are potentially previously unknown malware or other threats
opaque to state-of-the-art security tools, and in-depth host
inspection is necessary to determine the exact root cause.
A small number of the investigated “suspicious” incidents
were identified as policy violations (4.93%) or uncategorized
sites (8.64%). Their association with obscure software or
low-reputation sites prompted their identification as “suspicious” in the first-round investigation.
9. 4.4
Summary
With assistance from the enterprise SOC, we manually
investigated 784 Beehive incidents generated over the course
of two weeks. Overall, we find 25.25% of the incidents to be
malware-related or that warrant further SOC investigation,
39.41% to be policy violations, and 35.33% associated with
unrecognized, but automated, software or services. Only 8
of the 784 incidents (1.02%) were detected by existing stateof-the art security tools, demonstrating Beehive’s ability to
identify previously unknown anomalous behaviors.
5.
RELATED WORK
Network and host-based intrusion detection systems that
use statistical and machine learning techniques have seen
two decades of extensive research [11, 15, 33]. In addition,
both commercial and open source products that combine signature, traffic and anomaly inspection techniques are widely
available today [1, 2, 3]. To the best of our knowledge,
though, we present the first study of the challenges of sanitizing, correlating and analyzing large-scale log data collected from an enterprise of this scale, and of detecting both
compromised hosts and business policy violations.
There are many existing systems that aim to help human
analysts detect compromised hosts or stolen credentials, and
also a number of case studies on enterprise networks. For
example, Chapple et al. [12] present a case study on the
detection of anomalous authentication attempts to a university virtual private network using a clustering technique
focused on geographic distance. Zhang et al. [41] extend
this approach with additional machine-learning features to
automatically detect VPN account compromises in university networks. In an earlier study, Levine et al. [26] use
honeypots to detect exploited systems on enterprise networks. Similarly, Beehive aims to automate detection tasks
for security analysts. In addition to new detection methods, however, we present the largest-scale case study so far
on a real-life production network, together with the unique
challenges of analyzing big and disjoint log data.
While there exists work that analyzes a specific type of
malicious activity on the network (e.g., worms [37] or spammers [32]), recent attempts at malware detection mainly
focus on detection of botnets. For instance, several systems [10, 27, 35] use classification and correlation techniques
to identify C&C traffic from IRC botnets. BotTrack [16]
combines a NetFlow-based approach with the PageRank algorithm to detect P2P botnets. A number of studies monitor crowd communication behaviors of multiple hosts and
analyze the spatial-temporal correlations between them to
detect botnet infected hosts independent of the protocol
used for C&C communication [18, 20, 24, 40], while others
specifically focus on DNS traffic similarity [13, 36]. BotHunter [19] inspects the two-way communications at the network perimeter to identify specific stages of botnet infection. DISCLOSURE [8] presents a set of detection features
to identify C&C traffic using NetFlow records. Other works
focus on tracking and measuring botnets [4, 14, 17, 22, 34].
Other work aims to identify domains that exhibit malicious behavior instead of infected hosts. For example, some
systems [30, 29, 31, 21] propose various detection mechanisms for malicious fast-flux services. In a more general
approach, EXPOSURE [9], Notos [5], and Kopis [6] perform
passive DNS analysis to identify malicious domains. Ma et
al. [28] extract lexical and host-based URL features from
spam emails and use active DNS probing on domain names
to identify malicious websites. Another branch of research
aims to detect dynamically generated malicious domains by
modeling their lexical structures and utilizing the high number of failed DNS queries observed in botnets using such domains [7, 38, 39]. In comparison, Beehive uses standard log
data to detect suspicious activity in an enterprise network.
These approaches use network data for detection. In contrast, Beehive uses dirty enterprise log data to detect potentially malicious host behavior as well as policy violations
specific to an enterprise setting.
6.
CONCLUSIONS
In this paper, we presented a novel system called Beehive that attacks the problem of automatically mining and
extracting knowledge in a large enterprise from dirty logs
generated by a variety of network devices. The major challenges are the “big data” problem (at EMC, an average of 1.4
billion log messages—about 1 terabyte—are generated per
day), and the semantic gap between the information stored
in the logs and that required by security analysts to detect
suspicious behavior.
We developed efficient techniques to remove noise in the
logs, including normalizing log timestamps to UTC and creating an IP-to-hostname mapping that standardizes host
identifiers across log types. Using a custom whitelist built
by observing communications patterns in the enterprise, we
effectively reduced the data Beehive inspects from 300 million log messages per day to 80 million (a 74% reduction).
Beehive improves on signature-based approaches to detecting security incidents. Instead, it flags suspected security
incidents in hosts based on behavioral analysis. In our evaluation, Beehive detected malware infections and policy violations that went otherwise unnoticed by existing, stateof-the-art security tools and personnel. Specifically, for log
files collected in a large enterprise over two weeks, 25.25%
of Beehive incidents were confirmed to be malware-related
or to warrant further investigation by the enterprise SOC,
39.41% were policy violations, and 35.33% were associated
with unrecognized software or services.
To the best of our knowledge, ours is the first exploration
of the challenges of “big data” security analytics at the scale
of real-world enterprise log data.
Acknowledgments
We are grateful to members of the EMC CIRT team for
providing us access to the enterprise log data, and for their
help in investigating Beehive alerts.
This research is partly funded by National Science Foundation (NSF) under grant CNS-1116777. Engin Kirda also
thanks Sy and Laurie Sternberg for their generous support.
7.
REFERENCES
[1] OSSEC – Open Source Security.
http://www.ossec.net.
[2] Snort. http://www.snort.org.
[3] The Bro Network Security Monitor.
http://www.bro.org/.
[4] M. Abu Rajab, J. Zarfoss, F. Monrose, and A. Terzis.
A Multifaceted Approach to Understanding the
Botnet Phenomenon. In IMC, 2006.
10. [5] M. Antonakakis, R. Perdisci, D. Dagon, W. Lee, and
N. Feamster. Building a Dynamic Reputation System
for DNS. In USENIX Security, 2010.
[6] M. Antonakakis, R. Perdisci, W. Lee, N. Vasiloglou,
II, and D. Dagon. Detecting Malware Domains at the
Upper DNS Hierarchy. In USENIX Security, 2011.
[7] M. Antonakakis, R. Perdisci, Y. Nadji, N. Vasiloglou,
S. Abu-Nimeh, W. Lee, and D. Dagon. From
Throw-away Traffic to Bots: Detecting the Rise of
DGA-based Malware. In USENIX Security, 2012.
[8] L. Bilge, D. Balzarotti, W. Robertson, E. Kirda, and
C. Kruegel. Disclosure: Detecting Botnet Command
and Control Servers Through Large-scale NetFlow
Analysis. In ACSAC, 2012.
[9] L. Bilge, E. Kirda, K. Christopher, and M. Balduzzi.
EXPOSURE: Finding Malicious Domains Using
Passive DNS Analysis. In NDSS, 2011.
[10] J. R. Binkley and S. Singh. An Algorithm for
Anomaly-based Botnet Detection. In USENIX
SRUTI, 2006.
[11] D. Brauckhoff, X. Dimitropoulos, A. Wagner, and
K. Salamatian. Anomaly Extraction in Backbone
Networks Using Association Rules. In IMC, 2009.
[12] M. J. Chapple, N. Chawla, and A. Striegel.
Authentication Anomaly Detection: A Case Study on
a Virtual Private Network. In ACM MineNet, 2007.
[13] H. Choi, H. Lee, H. Lee, and H. Kim. Botnet
Detection by Monitoring Group Activities in DNS
Traffic. In IEEE CIT, 2007.
[14] E. Cooke, F. Jahanian, and D. McPherson. The
Zombie Roundup: Understanding, Detecting, and
Disrupting Botnets. In USENIX SRUTI, 2005.
[15] G. Dewaele, K. Fukuda, P. Borgnat, P. Abry, and
K. Cho. Extracting Hidden Anomalies Using Sketch
and non Gaussian Multiresolution Statistical
Detection Procedures. In ACM SIGCOMM LSAD,
2007.
[16] J. Fran¸ois, S. Wang, R. State, and T. Engel.
c
BotTrack: Tracking Botnets Using NetFlow and
PageRank. In IFIP TC 6 Networking Conf., 2011.
[17] F. C. Freiling, T. Holz, and G. Wicherski. Botnet
Tracking: Exploring a Root-cause Methodology to
Prevent Distributed Denial-of-service Attacks. In
ESORICS, 2005.
[18] G. Gu, R. Perdisci, J. Zhang, and W. Lee. BotMiner:
Clustering Analysis of Network Traffic for Protocoland Structure-independent Botnet Detection. In
USENIX Security, 2008.
[19] G. Gu, P. Porras, V. Yegneswaran, M. Fong, and
W. Lee. BotHunter: Detecting Malware Infection
Through IDS-driven Dialog Correlation. In USENIX
Security, 2007.
[20] G. Gu, J. Zhang, and W. Lee. BotSniffer: Detecting
Botnet Command and Control Channels in Network
Traffic. In NDSS, 2008.
[21] T. Holz, C. Gorecki, K. Rieck, and F. C. Freiling.
Measuring and Detecting Fast-Flux Service Networks.
In NDSS, 2008.
[22] J. P. John, A. Moshchuk, S. D. Gribble, and
A. Krishnamurthy. Studying Spamming Botnets Using
Botlab. In USENIX NSDI, 2009.
[23] I. T. Jolliffe. Principal Component Analysis.
Springer-Verlag, 1986.
[24] A. Karasaridis, B. Rexroad, and D. Hoeflin.
Wide-scale Botnet Detection and Characterization. In
USENIX HotBots, 2007.
[25] L. Kaufman and P. J. Rousseeuw. Finding Groups in
Data. An Introduction to Cluster Analysis. Wiley,
1990.
[26] J. Levine, R. LaBella, H. Owen, D. Contis, and
B. Culver. The Use of Honeynets to Detect Exploited
Systems Across Large Enterprise Networks. In IEEE
IAW, 2003.
[27] C. Livadas, R. Walsh, D. Lapsley, and W. Strayer.
Using Machine Learning Techniques to Identify
Botnet Traffic. In IEEE LCN, 2006.
[28] J. Ma, L. K. Saul, S. Savage, and G. M. Voelker.
Beyond Blacklists: Learning to Detect Malicious Web
Sites from Suspicious URLs. In ACM SIGKDD KDD,
2009.
[29] J. Nazario and T. Holz. As the Net Churns: Fast-flux
Botnet Observations. In MALWARE, 2008.
[30] E. Passerini, R. Paleari, L. Martignoni, and
D. Bruschi. FluXOR: Detecting and Monitoring
Fast-Flux Service Networks. In DIMVA, 2008.
[31] R. Perdisci, I. Corona, D. Dagon, and W. Lee.
Detecting Malicious Flux Service Networks through
Passive Analysis of Recursive DNS Traces. In ACSAC,
2009.
[32] A. Ramachandran and N. Feamster. Understanding
the Network-level Behavior of Spammers. In ACM
SIGCOMM, 2006.
[33] A. Sperotto, R. Sadre, and A. Pras. Anomaly
Characterization in Flow-Based Traffic Time Series. In
IEEE IPOM, 2008.
[34] B. Stone-Gross, M. Cova, L. Cavallaro, B. Gilbert,
M. Szydlowski, R. Kemmerer, C. Kruegel, and
G. Vigna. Your Botnet is My Botnet: Analysis of a
Botnet Takeover. In ACM CCS, 2009.
[35] W. Strayer, R. Walsh, C. Livadas, and D. Lapsley.
Detecting Botnets with Tight Command and Control.
In IEEE LCN, 2006.
[36] R. Villamar´
ın-Salom´n and J. C. Brustoloni. Bayesian
o
Bot Detection Based on DNS Traffic Similarity. In
ACM SAC, 2009.
[37] A. Wagner and B. Plattner. Entropy Based Worm and
Anomaly Detection in Fast IP Networks. In IEEE
WETICE, 2005.
[38] S. Yadav, A. K. K. Reddy, A. N. Reddy, and
S. Ranjan. Detecting Algorithmically Generated
Malicious Domain Names. In IMC, 2010.
[39] S. Yadav and A. N. Reddy. Winning With DNS
Failures: Strategies for Faster Botnet Detection. In
SECURECOMM, 2011.
[40] T.-F. Yen and M. K. Reiter. Traffic Aggregation for
Malware Detection. In DIMVA, 2008.
[41] J. Zhang, R. Berthier, W. Rhee, M. Bailey, P. Pal,
F. Jahanian, and W. H. Sanders. Safeguarding
Academic Accounts and Resources with the University
Credential Abuse Auditing System. In DSN, 2012.