The document discusses internet and network security risks and solutions. It provides an overview of common security threats like cybercrime, malware, and social engineering attacks. It then describes intrusion detection systems (IDS) and intrusion prevention systems (IPS) as basic concepts. IDS passively monitors network traffic and alerts administrators of potential threats, while IPS actively blocks malicious traffic in addition to detecting and alerting. The document analyzes IDS/IPS solutions and their role in providing security for networks and systems.
1. 1
INTRODUCTION
With all activities ,information and the vast amount of data ,the cyber
world give us almost unlimited freedom. However, there are risks. Because the
Internet is so easily accessible to anyone, it can be a dangerous place. Know
who you're dealing with or what you're getting into. Predators, cyber criminals,
bullies, and corrupt businesses will try to take advantage of the unwary visitor.
In February 2000, denial of service attacks against web giants like Yahoo
and eBay garnered a lot of attention from the media and from the Internet
community. When it comes to problems with Internet security, it is usually
major attacks against big companies that get the headlines. Unfortunately, many
small or home business owners do not realize that they are just as likely to be
targeted as any large company. As a consequence of existing in the digital age,
almost everyone is vulnerable to breaches of security. If your business relies on
computer or Internet technology, you need to be prepared to deal with security
issues.
Cyberspace is particularly difficult to secure due to a number of factors:
the ability of malicious actors to operate from anywhere in the world, the
linkages between cyberspace and physical systems, and the difficulty of
reducing vulnerabilities and consequences in complex cyber networks. Of
growing concern is the cyber threat to critical infrastructure, which is
increasingly subject to sophisticated cyber intrusions that pose new risks. As
information technology becomes increasingly integrated with physical
infrastructure operations, there is increased risk for wide scale or high-
consequence events that could cause harm or disrupt services of everyday life.
2. 2
1 OVERVIEW of SECURITY RISKS AND PROTECTION
TECHNOLOGIES
1.1 OVERVIEW OF THREATS AND RISKS IN THE CYBER WORLD
Cyber risks can be divided into three distinct areas:
• Cyber crime Conducted by individuals working alone, or in organised groups,
intent on extracting money, data or causing disruption, cyber crime can
take many forms, including the acquisition of credit/debit card data and
intellectual property, and impairing the operations of a website or service.
• Cyber war A nation state conducting sabotage and espionage against another
nation in order to cause disruption or to extract data. This could involve
the use of Advanced Persistent Threats (APTs).
• Cyber terror
• An organisation, working independently of a nation state, conducting terrorist
activities through the medium of cyberspace.
Organisations that have to consider measures against cyber war or cyber terror
include governments, those within the critical national infrastructure, and very
high-profile institutions. It is unlikely that most organisations will face the threat
of cyber war or cyber terror.
Congruent with the rapid pace of technological change, the world of cyber crime
never stops innovating either. Every month, Microsoft publishes a bulletin of the
vulnerabilities of its systems, an ever-growing list of known threats.
3. 3
Types of malware
Cyber criminals operate remotely, in what is called ‘automation at a distance’,
using numerous means of attack available, which broadly fall under the umbrella
term of malware (malicious software). These include:
• Viruses
Aim: Gain access to, steal, modify and/or corrupt information and files
from a targeted computer system.
Technique: A small piece of software program that can replicate itself and
spread from one computer to another by attaching itself to another
computer file.
• Worms
Aim: By exploiting weaknesses in operating systems, worms seek to
damage networks and often deliver payloads which allow remote control
of the infected computer.
Technique: Worms are self-replicating and do not require a program to
attach themselves to. Worms continually look for vulnerabilities and report
back to the worm author when weaknesses are discovered.
• Spyware/Adware
Aim: To take control of your computer and/or to collect personal
information without your knowledge.
Technique: By opening attachments, clicking links or downloading
infected software, spyware/adware is installed on your computer.
4. 4
• Trojans
Aim: To create a ‘backdoor’ on your computer by which information can
be stolen and damage caused.
Technique: A software program appears to perform one function (for
example, virus removal) but actually acts as something else.
Attack vectors
There are also a number of attack vectors available to cyber criminals
which allow them to infect computers with malware or to harvest stolen
data:
• Phishing
An attempt to acquire users’ information by masquerading as a legitimate
entity. Examples include spoof emails and websites. See ‘social
engineering’ below.
• Pharming
An attack to redirect a website’s traffic to a different, fake website, where
the individuals’ information is then compromised. See ‘social engineering’
below.
• Drive-by
Opportunistic attacks against specific weaknesses within a system.
• MITM ‘Man in the middle attack’ where a middleman impersonates each
endpoint and is thus able to manipulate both victims.
5. 5
• Social engineering Exploiting the weakness of the individual by making them
click malicious links, or by physically gaining access to a computer
through deception. Pharming and phishing are examples of social
engineering
Spyware is software that aims to gather information about a person or
organization without their knowledge and that may send such information
to another entity without the consumer's consent, or that asserts control
over a computer without the consumer's knowledge.[1]
"Spyware" is mostly classified into four types: system monitors, trojans,
adware, and tracking cookies.[2] Spyware is mostly used for the purposes of
tracking and storing Internet users' movements on the Web and serving up pop-
up ads to Internet users. whenever spyware is used for malicious purposes, its
presence is typically hidden from the user and can be difficult to detect. Some
spyware, such as keyloggers, may be installed by the owner of a shared,
corporate, or public computer intentionally in order to monitor users.
While the term spyware suggests software that monitors a user's
computing, the functions of spyware can extend beyond simple monitoring.
Spyware can collect almost any type of data, including personal information like
internet surfing habits, user logins, and bank or credit account information.
Spyware can also interfere with user control of a computer by installing
additional software or redirecting web browsers. Some spyware can change
computer settings, which can result in slow Internet connection speeds, un-
authorized changes in browser settings, or changes to software settings.
Cyber crime is only likely to increase, despite the best efforts of government
agencies and cyber security experts. Its growth is being driven by the
6. 6
expanding number of services available online and the increasing
sophistication of cyber criminals who are engaged in a cat-and-mouse game
with security experts.
Attackers, Hackers and Crackers any time a large attack is reported in
the media, there is a great deal of speculation about who perpetrated the attack
and why. By now, most people have heard the term hacker bandied about by
the media. Often attacks are blamed on these so-called hackers. Who or what
are hackers? What role do they play in Internet security and what motivates
them to do what they do?
Hackers: The term hacker was originally used to refer to a self-taught
computer expert who is highly skilled with technology, programming, and
hardware. Many hackers employ these skills to test the strength and integrity of
computer systems for a wide variety of reasons: to prove their own ability, to
satisfy their curiosity about how different programs work, or to improve their
own programming skills by exploring the programming of others. The term
hacker has been adopted by the mass media to refer to all people who break into
computer systems, regardless of motivation; however, in the media the term
hacker is often associated with people who hack illegally for criminal purposes.
Many in the Internet security community strongly disagree with this use of the
term.
7. 7
Crackers People within the Internet community tend to refer to people
who engage in unlawful or damaging hacking as crackers, short for ?criminal
hackers?. The term cracker generally connotes a hacker who uses his or her
skills to commit unlawful acts, or to deliberately create mischief. Unlike
hackers whose motivations may be professional or community enhancement,
the motivation of crackers is generally to cause mischief, create damage or to
pursue illegal activities, such as data theft, or vandalism.
1.2 SECURITY IN CYBER WORLD
Cyber security, also referred to as information technology security,
focuses on protecting computers, networks, programs and data from unintended
or unauthorized access, change or destruction.
Internet security can be defined as the protection of data from theft, loss
or unauthorized access, use or modification. With the constantly evolving nature
of the Internet, it is vital that users continuously protect themselves and their
information. This issue is so important that many large firms employ full-time
security experts or analysts to maintain network security. However, few, if any,
home and small business owners can afford that luxury. Therefore it is up to
small-office users to take these issues into their own hands.
Internet security relies on specific resources and standards for protecting
data that gets sent through the Internet. This includes various kinds of
encryption such as Pretty Good Privacy (PGP). Other aspects of a secure Web
setup includes firewalls, which block unwanted traffic, and anti-malware, anti-
8. 8
spyware and anti-virus programs that work from specific networks or devices to
monitor Internet traffic for dangerous attachments.
Internet security is generally becoming a top priority for both businesses
and governments. Good Internet security protects financial details and much
more of what is handled by a business or agency’s servers and network
hardware. Insufficient Internet security can threaten to collapse an e-commerce
business or any other operation where data gets routed over the Web.
To understand What is network security?, it helps to understand that no
single solution protects you from a variety of threats. You need multiple layers
of security. If one fails, others still stand.
Network security is accomplished through hardware and software. The
software must be constantly updated and managed to protect you from emerging
threats.A network security system usually consists of many components. Ideally,
all components work together, which minimizes maintenance and improves
security.
Network security is accomplished through hardware and software. The
software must be constantly updated and managed to protect you from emerging
threats.
A network security system usually consists of many components. Ideally,
all components work together, which minimizes maintenance and improves
security.
Network security components often include:
9. 9
Anti-virus and anti-spyware
Firewall, to block unauthorized access to your network
Intrusion prevention systems (IPS), to identify fast-spreading threats, such as
zero-day or zero-hour attacks
Virtual Private Networks (VPNs), to provide secure remote access.
With network security in place, your company will experience many
business benefits. Your company is protected against business disruption, which
helps keep employees productive. Network security helps your company meet
mandatory regulatory compliance. Because network security helps protect your
customers' data, it reduces the risk of legal action from data theft.
Ultimately, network security helps protect a business's reputation, which
is one of its most important assets.
Network outages, data compromised by hackers, computer viruses and
other incidents affect our lives in ways that range from inconvenient to life-
threatening. As the number of mobile users, digital applications and data
networks increase, so do the opportunities for exploitation.
Layered security is the key to protecting any size network, and for most
companies, that means deploying both intrusion detection systems (IDS) and
intrusion prevention systems (IPS).
10. 10
1.3 BASIC CONCEPT OF IDS/IPS
Used in computer security, intrusion detection refers to the process of
monitoring computer and network activities and analyzing those events to look
for signs of intrusion in your system. The point of looking for unauthorized
intrusions is to alert IT professionals and system administrators within your
organization to potential system or network security threats and weaknesses.
While Intrusion Detection and Intrusion Prevention Systems (IDS/IPS) have
been around for decades, the definition of what they are tasked with and how
they perform their functions has evolved, just as the threats facing organizations
today have evolved. Originally, IDS platforms were tasked with monitoring
communications and providing a method of alerting staff to attacks that where
being detected on the network (typically out of band) so that further action could
be taken to stop them. The evolution into IPS included a method of
implementing devices differently, including the ability to detect attacks and to
take some action to stop them automatically. This was traditionally implemented
through in-band sensors or appliances that were configured with an ever-
growing list of known threat signatures.
When it comes to IPS and IDS, it's not a question of which technology to
add to your security infrastructure - both are required for maximum protection
against malicious traffic. In fact, vendors are increasingly combining the two
technologies into a single box.
At its most basic, an IDS device is passive, watching packets of data
traverse the network from a monitoring port, comparing the traffic to configured
rules, and setting off an alarm if it detects anything suspicious. An IDS can
11. 11
detect several types of malicious traffic that would slip by a typical firewall,
including network attacks against services, data-driven attacks on applications,
host-based attacks like unauthorized logins, and malware like viruses, Trojan
horses, and worms. Most IDS products use several methods to detect threats,
usually signature-based detection, anomaly-based detection, and stateful
protocolanalysis.
The IDS engine records the incidents that are logged by the IDS sensors in a
database and generates the alerts it sends to the network administrator. Because
IDS gives deep visibility into network activity, it can also be used to help
pinpoint problems with an organization's security policy, document existing
threats, and discourage users from violating an organization'ssecuritypolicy.
IDPSs are able to monitor the events of interests on the systems and/or
networks and are then able to identify possible incidents, log information about
them, and attempt to stop common attacks and report them to security
administrators. In the past, Intrusion Detection and Prevention (IDPS) has either
been signature-based (able to check activity against known attackers’ patterns,
the signature), anomaly-based (also referred to as heuristic, that alerts when
traffic and activity are not normal), or based on stateful protocol analysis that
looks at the “state” in a connection and “remembers” significant events that
occur.
The primary complaint with IDS is the number of false positives the
technology is prone to spitting out - some legitimate traffic is inevitable tagged
as bad. The trick is tuning the device to maximize its accuracy in recognizing
true threats while minimizing the number of false positives; these devices should
be regularly tuned as new threats are discovered and the network structure is
12. 12
altered. As the technology has matured in the last several years, it has gotten
better at weeding out false positives. However, completely eliminating them
while still maintaining strict controls is next to impossible - even for IPS, which
some consider the next step in the evolution of IDS.
13. 13
.2. ANALYSISOF IDSIPS SOLUTIONS
2.1 UNDERSTANDING IDS/IPS
IPS and IDS systems look for intrusions and symptoms within traffic.
IPS/IDS systems would monitor for unusual behavior, abnormal traffic,
malicious coding and anything that would look like an intrusion by a hacker
being attempted.
IPS (Intrusion Prevention System) systems are deployed inline and actually
take action by blocking the attack, as well as logging the attack and adding the
source IP address to the block list for a limited amount of time; or even
permanently blocking the address depending on the defined settings. Hackers
take part in lots of port scans and address scans, intending to find loop holes
within organizations. IPS systems would recognize these types of scans and take
actions such as block, drop, quarantine and log traffic. However this is the basic
functionality of IPS. IPS systems have many advanced capabilities in sensing
and stopping such attacks.
IDS (Intrusion Detection System) systems only detect an intrusion, log the
attack and send an alert to the administrator. IDS systems do not slow networks
down like IPS as they are not inline.
You may wonder why a company would purchase an IDS over an IPS?
Surely a company would want a system to take action and block such attacks
rather than letting it pass and only logging and alerting the administer. Well
there’s a few reasons; however there are two primary reasons which stand out.
IDS systems if not fine tuned, just like IPS will also produce false positives.
However it would be very annoying to have an IPS system producing false
14. 14
positives as legitimate network traffic will be blocked as where an IDS will just
send alerts and log the false attack. The 2nd reason is some administrators and
managers do not want a system to take over and make decisions on their behalf;
they would rather receive an alert and look into the problem and take action
themselves.
However that said today you will find solutions with both capabilities of
IDS and IPS built in. IDS can be used initially to see how the system behaves
without actually blocking anything. Then once fine tuned IPS can be turned on
and the system can be deployed inline to provide full protection.
IDS — A Passive Security Solution
An intrusion detection system (IDS) is designed to monitor all inbound and
outbound network activity and identify any suspicious patterns that may indicate
a network or system attack from someone attempting to break into or
compromise a system. IDS is considered to be a passive-monitoring system,
since the main function of an IDS product is to warn you of suspicious activity
taking place − not prevent them. An IDS essentially reviews your network
traffic and data and will identify probes, attacks, exploits and other
vulnerabilities. IDSs can respond to the suspicious event in one of several ways,
which includes displaying an alert,logging the event or even paging an
administrator. In some cases the IDS may be prompted to reconfigure the
network to reduce the effects of the suspicious intrusion.
An IDS specifically looks for suspicious activity and events that might be
the result of a virus, worm or hacker. This is done by looking for known
intrusion signatures or attack signatures that characterize different worms or
viruses and by tracking general variances which differ from regular system
activity. The IDS is able to provide notification of only known attacks.The term
IDS actually covers a large variety of products, for which all produce the end
15. 15
result of detecting intrusions. An IDS solution can come in the form of cheaper
shareware or freely distributed open source programs, to a much more expensive
and secure vendor software solution. Additionally, some IDSs consist of both
software applications and hardware appliances and sensor devices which are
installed at different points along your network.
IPS — An Active Security Solution : IPS or intrusion prevention system,
is definitely the next level of security technology with its capability to provide
security at all system levels from the operating system kernel to network data
packets. It provides policies and rules for network traffic along with an IDS for
alerting system or network administrators to suspicious traffic, but allows the
administrator to provide the action upon being alerted. Where IDS informs of a
potential attack, an IPS makes attempts to stop it. Another huge leap over IDS,
is that IPS has the capability of being able to prevent known intrusion
signatures, but also some unknown attacks due to its database of generic attack
behaviors. Thought of as a combination of IDS and an application layer firewall
for protection, IPS is generally considered to be the "next generation" of IDS.
Currently, there are two types of IPSs that are similar in nature to IDS. They
consist of host-based intrusion prevention systems (HIPS) products and
network-based intrusion prevention systems(NIPS).
16. 16
2.2 METHODS OF IDS/IPS
There are a few different types of intrusion systems. Firstly there’s host
based (HIDS) and network based (NIDS). Network based (NIDS) monitors for
intrusions on the network. Host based sits on a computer itself and monitors the
host itself. HIDS are expensive to deploy on all computers, and so are used for
servers that require this extra protection, where network based is usually cheaper
to purchase as the investment is in one appliance sitting on your network
monitoring traffic.
Intrusion detection systems are network or host based solutions.
Network-based IDS systems (NIDS) are often standalone hardware appliances
that include network intrusion detection capabilities. It will usually consist of
hardware sensors located at various points along the network or software that is
installed to system computers connected to your network, which analyzes data
packets entering and leaving the network. Host-based IDS systems (HIDS) do
not offer true real-time detection, but if configured correctly are close to true
real-time.
Host-based IDS systems consist of software agents installed on individual
computers within the system. HIDS analyze the traffic to and from the specific
computer on which the intrusion detection software is installed on. HIDS
systems often provide features you can't get with a network-based IDS. For
example, HIDS are able to monitor activities that only an administrator should
be able to implement. It is also able to monitor changes to key system files and
any attempt to overwrite these files. Attempts to install Trojans or backdoors can
also be monitored by a HIDS and stopped. These specific intrusion events are
While it depends on the size of your network and the number of individual
computers which require intrusion detection system, NIDS are usually a cheaper
17. 17
solution to implement and it requires less administration and training − but it is
not as versatile as a HID. Both systems will require Internet access (bandwidth)
to ensure the system is kept up-to-date with the latest virus and worm signatures.
HIDS and NIDS can come in a number of types of intrusion systems as well.
Signature based Signatures are created by vendors based on potential
attacks and attacks that have been taken place in the past. These signatures are
scheduled and downloaded by the intrusion software itself. Any packets arriving
into the network are compared to the set of downloaded signatures comparing
these for any attacks. Signature based systems are the most common. Most UTM
appliances consist of signature based intrusion prevention/detection systems.
The only downfall to these systems is that they can not detect new attacks, as
they only compare attacks to the signatures their system currently holds.
Anomaly based In anomaly based, the system would first need to learn the
NORMAL behavior, traffic or protocol set of the network. When the system has
learnt the normal state of a network and the types of packets and throughput it
handles on a daily basis, taking into account peak times such as lunch time for
example for web browsing, then it can be put into action. Now when traffic is
detected that is out of the normal state of the network, the anomaly based
detection system would take action.
The good thing about this type of system is that it can detect new attacks;
it does not need to rely on signatures. The bad thing is if you do not spend time
fine stunning the system and maintaining it, it will usually produce many false
positives (Stop normal traffic). Also some clever hackers try and emulating their
attacks as normal traffic, however this is usually difficult to do from a hacking
perspective, but if they get it right, it may fool the ADS system as normal and
legitimate traffic.
18. 18
Rule based Rule based systems are more advanced and cleverly built
systems. A knowledge base programmed as rules will decide the output
alongside an inference engine. If the defined rules for example all match, a
certain assumption can be determined in which an action may take place. This
assumption is the power of the inference engine. The inference engine can
assume an attack may be occurring because of so many factors; this is unique
and is very much behaving like the human mind. In normal computing
assumptions can not be made, its either yes or no, but the inference engine adds
a different level of thinking; it also adds the “Probably” to the list, like humans.
If it rains and is warm, we can assume it may thunder. If more traffic was
leaving the company than usual, as well as coming from a certain server, the
inference engine may assume, the server could be compromised by a hacker.
Many IDS/IPS solutions have combined both signature and anomaly based
detection system.
2.3 BEST IDS/IPS SOLUTIONS
Most technologies for detecting attacks and other malicious and unwanted
behavior concentrate on one type of malicious activity, such as antivirus
software targeting malware. What makes intrusion prevention systems unique is
they have the ability to detect many different types of activity at all levels of the
network stack, including malicious behavior by or within thousands of
application protocols.
Today's network intrusion prevention systems are available in three main forms:
• Dedicated -- either hardware-based appliances or virtual appliances dedicated
to IPS functions only;
• Integrated -- generally a module enabled on another enterprise security control,
19. 19
especially a next-generation firewall (NGFW); and
• Cloud-based -- available as a service from a cloud-based IPS provider.
This article, the last in this series, examines the best intrusion prevention
systems on the market today. It is difficult to compare them across these three
forms because each form is best suited to certain cases and conditions, as
explained in the first article in this series. For the purposes of simplifying and
focusing the comparison, this article looks at dedicated IPS products only.
Although hardware-based appliances and virtual appliances have some inherent
differences because of their forms, in most cases, their functionality is nearly
identical.
The best intrusion prevention systems available today, according to the IPS
products studied for this article, are:
• Cisco FirePOWER and its virtual appliance version, Cisco Virtual Next-
Generation IPS;
• HP N Platform Next-Generation Intrusion Prevention System (NGIPS) and HP
TippingPoint NX Next-Generation Intrusion Prevention System;
• IBM Security Network Intrusion Prevention System;
• McAfee Network Security Platform (NSP), which is available in three forms:
M Series, NS Series and virtual sensor; and
• Radware DefensePro.
These products were evaluated using public sources of information, such as
product websites, white papers and product manuals. IPS criteria used for the
evaluation are as follows:
• Criterion 1: How broad and comprehensive the IPS's detection capabilities are
• Criterion 2: How well the IPS can incorporate an understanding of context to
improve its functioning
• Criterion 3: How effectively the IPS can use threat intelligence feeds
20. 20
These three criteria are meant to be only a small part of a much larger IPS
evaluation process. Every organization has a unique environment, unique
security requirements, and unique risk tolerance characteristics. Consider the
rest of this article as input for an evaluation that should be considered, along
with many other inputs. If an evaluation includes integrated and/or cloud-based
forms of IPS, as well as dedicated technologies, these criteria may be helpful,
but consider that additional criteria will be needed to compare across IPS forms.
Uses a wide range of techniques to detect attacks Examples of common
techniques include signature- or anomaly-based detection, network flow or
behavior analysis, denial-of-service detection, and deep-packet inspection. All
major IPSes use multiple techniques, because each technique detects a
somewhat different set of attacks, but some IPSes use several techniques to
provide the broadest attack detection possible. The products that claim the
largest range of detection techniques are IBM Security Network Intrusion
Prevention System, Intel Security McAfee NSP and Radware DefensePro. This
doesn't necessarily mean other products have a narrow range, only that those
products do not specifically claim a wide range.
Detects zero-day attacks and other attacks that have never been seen An
IPS's ability to understand the security implications of completely new attacks
has become a key component to its detecting and stopping attacks that most
other security controls cannot recognize. All the IPS products studied for this
article have this ability to some extent because they can detect aberrations in
expected behavior. Ideally an IPS also performs extensive protocol analysis to
find potential exploitation attempts of both known and unknown vulnerabilities
in those protocols. Both the HP TippingPoint NGIPS and the IBM Security
Network Intrusion Prevention System specify their support for this capability.
Choosing the best intrusion prevention system It is important to do your
21. 21
own evaluation before selecting the best intrusion prevention system for your
organization. The first step is to determine which form or combination of forms
of IPS -- dedicated, integrated or cloud-based -- best suits its needs. If the
selected forms include dedicated products, then look at the products studied in
this article, and potentially others as well, in terms of the criteria defined in this
feature, as well as many other criteria.
2.4 NEXT GENERATION IDSIPS SOULTION
Traditional Intrusion Detection and Intrusion Prevention Systems
(IDS/IPS) have evolved into the Next Generation Intrusion Prevention Systems
(NGIPS). See what the new breed of IPS has to offer and how the concept
works.
The new breed of IPS takes advantage of the traditional Intrusion Prevention
Systems but adds a number of functionalities that allow it to provide better
protection for modern organizational networks and devices. Some of these
added functionalities include:
Network Awareness -- provides a knowledge of the devices that exist on
the network. This is very valuable information when gathered in both
small and large quantities. It allows an organization to have the ability to
know the types of devices (OS, device types, etc) that exist on the network
and be able to pick out and highlight those that are outside the norm. Any
device types that are not considered normal will be flagged and alerts can
be configured to notify the appropriate individuals. This also typically
22. 22
extends into the detection of which software packages are being used to
generate the traffic on the network.
Application Awareness -- provides the ability to pick out and highlight
applications that are being run on the network and the users that are
running them. This capability allows policies to be created to control
which applications are allowed and which are not, by whom and to what
level (e.g. Facebook, Jabber, Skype, Twitter, Youtube, etc).
Identity Awareness -- provides the ability to gather identity information
for the devices and applications that are attached to the network and for the
traffic that is being transmitted. This information can be gathered using a
number of different techniques and databases, such as Microsoft Active
Directory (AD) and LDAP.
Behavior Awareness -- provides the ability to establish and monitor the
baseline behavior of network devices. This information is then used to
contrast against continued usage patterns. Anything that stands out will be
reported and/or mitigated by policy (e.g. bandwidth consumption,
performance degradation, etc).
Real Time Automated Response -- provides the ability to respond to
events as they occur and react with the appropriate response based on
policy.
Automatic IPS Tuning -- provides the ability for a platform to dynamically
tune itself based on the information gathered. This reduces the amount of
interactive engineer time that is needed to alter rules to the conditions.
Examples of this include the enabling or disabling of certain scanning
signatures or techniques based on the discovered operating systems being
used or applications being run.
23. 23
It is important to note that while the features of a NGIPS are very
important to implement on a network, it should not be considered a complete
solution for system protection. NGIPS solutions are typically implemented
either as a point product (where the only thing the appliance does is IPS) or as a
combined solution with other features and options. A complete security solution
will require that organizations have a multi-tiered approach to systems security.
This includes the implementation of a number of different solutions that each
work in combination with each other.
It is important that the solutions that are selected (NGIPS or otherwise),
each have the ability to integrate into a combined management and/or
monitoring system and hopefully with each other. This allows security staff to
quickly view all of the information from multiple solutions to gain the most
comprehensive view of the network and the devices attached to it. It also
provides the ability for multiple solutions to be integrated into each other. For
example, if an AMP solution finds a new malware and indicates that it uses a
specific unique port number and/or protocol, it can be integrated with a firewall
solution to automatically block it before it gains access into the organizational
network parameter.
Its being estimated that by 2020, 60 percent of enterprise information
security budgets will be allocated for rapid detection and response approaches
and by 2018 80 percent of endpoint protection platforms will include user
activity monitoring and forensic capabilities. This follows the evolution of the
Next Generation Intrusion Prevention Systems. These platforms will continue to
transition into smarter, more capable tools and because of this they will grow
even more dynamic as malicious attacks evolve.
IPS/IDS has changed, as research shows, with AI techniques that have
improved IDSs by making them capable of detecting both current and future
24. 24
intrusion attacks while triggering fewer false positives and negatives. New
ANNIDS (Neural networks applied to IDS) techniques have been able to
improve the way detection systems are trained to recognize patterns, conduct
problem solving and fault diagnosis too.
25. 25
3. A NEW METHOD FOR LARGE SCALE NETWORK PROTECTION(IIDSIPS) OF
ENTERPRISER ENVIRONMENT
3.1 IIDSPS( INTELLIGENT INTRUSION DETECTION AND PREVENTION
SYSTEM)
Intrusion systems have been the subject of considerable research for
decades to improve the inconsistencies and inadequacies of existing methods,
from basic detect ability of an attack to the prevention of computer misuse. It
remains a challenge still today to detect and classify known and unknown
malicious network activities through identification of intrusive behavioral
patterns (anomaly detection) or pattern matching (misuse or signature-based
detection). Meanwhile, the number of network attack incidents continues to
grow.
Protecting a computer network against attacks or cybersecurity threats is
imperative, especially for companies that need to protect not only their own
business data but also sensitive information of their clients as well as of their
employees. It is not hard to see why even just one breach in data security from a
single intrusion of a computer network could wreak havoc on the entire
organization. Not only would it question the reliability of the networks’
infrastructure, but it could also seriously damage the business’s reputation.
An organization’s first defense against breaches is a well-defined
corporate policy and management of systems, as well as the involvement of
users in protecting the confidentiality, integrity, and availability of all
information assets. Security awareness training is a baseline for staff to gain the
knowledge necessary to deter computer breaches and viruses, mitigate the risks
26. 26
associated with malicious attacks, and defend against constantly evolving
threats.
Users’ awareness and strict IT policies and procedures can help defend a
company from attacks, but when a malicious intrusion is attempted, technology
is what helps systems administrators protect IT assets. When it comes to
perimeter data security, traditional defense mechanisms should be in layers:
firewalls, intrusion detection systems (IDS) and intrusion prevention systems
(IPS) can be used.
Research and new developments in the field of IDPS (Intrusion Detection
and Prevention System) prove different approaches to anomaly and misuse
detection can work effectively in practical settings, even without the need of
human interaction/supervision in the process.
Several case studies emphasize that the use of Artificial Neural Networks
(ANN) can establish general patterns and identify attack characteristics in
situations where rules are not known. A neural network approach can adapt to
certain constraints, learn system characteristics, recognize patterns and compare
recent user actions to the usual behavior; this allows resolving many
issues/problems even without human intervention. The technology promises to
detect misuse and improve the recognition of malicious events with more
consistency. A neural network is able to detect any instances of possible misuse,
allowing system administrators to protect their entire organization through
enhanced resilience against threats.
27. 27
3.2 THE NURAL NETWORK
What are Artificial Neural Networks (ANNs)?
The inventor of the first neurocomputer, Dr. Robert Hecht-Nielsen, defines a
neural network as −
"...a computing system made up of a number of simple, highly interconnected
processing elements, which process information by their dynamic state response
to external inputs.”
Basic Structure of ANNs
Fig 3.1.Human brain neuron.
The idea of ANNs is based on the belief that working of human brain by making
the right connections, can be imitated using silicon and wires as living neurons
and dendrites. The human brain is composed of 100 billion nerve cells called
neurons. They are
connected to other thousand cells by Axons. Stimuli from external
28. 28
environment or inputs from sensory organs are accepted by dendrites. These
inputs create electric impulses, which quickly travel through the neural network.
A neuron can then send the message to other neuron to handle the issue or does
not send it forward.
ANNs are composed of multiple nodes, which imitate biological neurons
of human brain. The neurons are connected by links and they interact with each
other. The nodes can take input data and perform simple operations on the data.
The result of these operations is passed to other neurons. The output at each
node is called its activation or node value.
Each link is associated with weight. ANNs are capable of learning, which takes
place by altering weight values. The following illustration shows a simple ANN
neuron.
Fig 3.2.Basic artificial neuron example
A set of input values (Xn) and associated weights (Wnj)
29. 29
A function (transfer function) that sums the weights and maps the results
to an output (Oj activation).
Transfer (Activation) Functions
The transfer function translates the input signals to output signals.
Four types of transfer functions are commonly used, Unit step (threshold),
sigmoid, piecewise linear, and Gaussian.
In my example am going to use the Sigmoid Function Activation function(threshold)
The output is set at one of two levels, depending on whether the total input is greater
than or less than some threshold value
Examples of transfer function(Fig.1) and threshold (unit step)(Fig.2) is shown below:
Fig.1 Fig.2
Fig 3.3 graph of non leanr and sigmoid functions
With each single neuron provided with a different significant and yet
simple mathematical operation or a function so to speak, and all the results from
30. 30
the output data linked together to be evaluated and for the most desired
outcome, and this is all one giant network of neurons.
Working of ANNs In the topology diagrams shown, each arrow represents
a connection between two neurons and indicates the pathway for the flow of
information. Each connection has a weight, an integer number that controls the
signal between the two neurons.
If the network generates a “good or desired” output, there is no need to adjust
the weights. However, if the network generates a “poor or undesired” output or
an error, then the system alters the weights in order to improve subsequent
results.
Types of Artificial Neural Networks There are two Artificial Neural Network
topologies. Feedforward Back-Propagation and SOM( Kohonen self-organized
Map).
Feedforward Back-Propagation The feedforward, back-propagation
architecture was developed in the early 1970's by several independent sources
(Werbor; Parker; Rumelhart, Hinton and Williams). This independent co-
development was the result of a proliferation of articles and talks at various
conferences which stimulated the entire industry. Currently, this synergistically
developed back-propagation architecture is the most popular, effective, and
easy-to-learn model for complex, multi-layered networks. Its greatest strength is
in non-linear solutions to ill-defined problems. The typical back-propagation
network has an input layer, an output layer, and at least one hidden layer. There
is no theoretical limit on the number of hidden layers but typically there are just
one or two. Some work has been done which indicates that a maximum of five
layers (one input layer, three hidden layers and an output layer) are required to
solve problems of any complexity. Each layer is fully connected to the
31. 31
succeeding layer.
An abbreviation for "backward propagation of errors", is a common
method of training artificial neural networks used in conjunction with an
optimization method such as gradient descent. The method calculates the
gradient of a loss function with respect to all the weights in the network. The
gradient is fed to the optimization method which in turn uses it to update the
weights, in an attempt to minimize the loss function.
Backpropagation requires a known, desired output for each input value in
order to calculate the loss function gradient. It is therefore usually considered to
be a supervised learning method, although it is also used in some unsupervised
networks such as autoencoders. It is a generalization of the delta rule to multi-
layered feedforward networks, made possible by using the chain rule to
iteratively compute gradients for each layer. Backpropagation requires that the
activation function used by the artificial neurons (or "nodes") be differentiable.
The information flow is unidirectional. A unit sends information to other unit
from which it does not receive any information. There are no feedback loops.
They are used in pattern generation/recognition/classification. They have fixed
inputs and outputs.
As noted above, the training process normally uses some variant of the Delta
Rule, which starts with the calculated difference between the actual outputs and
the desired outputs. Using this error, connection weights are increased in
proportion to the error times a scaling factor for global accuracy. Doing this for
an individual node means that the inputs, the output, and the desired output all
have to be present at the same processing element. The complex part of this
32. 32
learning mechanism is for the system to determine which input contributed the
most to an incorrect output and how does that element get changed to correct the
error. An inactive node would not contribute to the error and would have no
need to change its weights. To solve this problem, training inputs are applied to
the input layer of the network, and desired outputs are compared at the output
layer. During the learning process, a forward sweep is made through the
network, and the output of each element is computed layer by layer. The
difference between the output of the final layer and the desired output is back-
propagated to the previous layer(s), usually modified by the derivative of the
transfer function, and the connection weights are normally adjusted using the
Delta Rule. This process proceeds for the previous layer(s) until the input layer
is reached. MLP(Multi-level perceptron) is a prime example of this kind of
ANN.
The Perceptron was first introduced by F. Rosenblatt in 1958.
It is a very simple neural net type with two neuron layers that accepts
only binary input and output values (0 or 1). The learning process is supervised
and the net is able to solve basic logical operations like AND or OR. It is also
used for pattern classification purposes.
More complicated logical operations (like the XOR problem) cannot be solved
by a Perceptron.
Multi-layer Perceptron (MLP) :It is a freedforward artificial neural
network model made for pattern recognition, It gives out a set of appropriate
outputs. MLP consist of multiple layers, is a supervised learning algorithm that
learns a function by training on a dataset, where “X” is the number of
dimensions for input and “m”is the the number of dimensions for output. Given
a set of features “X = x1+x2…,xm” and a target , it can learn a non-linear
33. 33
function approximator for either classification or regression. It is different from
logistic regression, in that between the input and the output layer, there can be
one or more non-linear layers, called hidden layers.. Figure 1 shows a one
hidden layer MLP with scalar output
Hidden layer
Feature(X)
Output
Figure 3.4 . One hidden layer MLP.
The leftmost layer, known as the input layer, consists of a set of neurons {xi|x1,x2…,xm}
representing the input features. Each neuron in the hidden layer transforms the values
from the previous layer with a weighted linear summation”w1x1+w2x2+…+wm+xm”,
followed by a non-linear activation function g(.): RR - like the hyperbolic tan function.
X3
X1
+1
X2
Xm
Ak
A2
A1
+1
f(X)
34. 34
The output layer receives the values from the last hidden layer and transforms them into
output values.
Kohonen Self-organizing Map self-organizing map (SOM) or self-
organising feature map (SOFM) is a type of artificial neural network (ANN) that
is trained using unsupervised learning to produce a low-dimensional (typically
two-dimensional), discretized representation of the input space of the training
samples, called a map. Self-organizing maps are different from other artificial
neural networks as they apply competitive learning as opposed to error-
correction learning (such as backpropagation with gradient descent), and in the
sense that they use a neighborhood function to preserve the topological
properties of the input space.
A self-organizing map consists of components called nodes or neurons.
Associated with each node are a weight vector of the same dimension as the
input data vectors, and a position in the map space. The usual arrangement of
nodes is a two-dimensional regular spacing in a hexagonal or rectangular grid.
The self-organizing map describes a mapping from a higher-dimensional input
space to a lower-dimensional map space. The procedure for placing a vector
from data space onto the map is to find the node with the closest (smallest
distance metric) weight vector to the data space vector
In summation, an ANN consist of treatments to transform a set of inputs to a set
of searched outputs, though a set of simple processing units. Also nodes and
connection between them .Subnets are input nodes , output nodes and nodes
between them are a layer or can be multiple layers of processing nodes.the
connection between the nodes are associated with weights and are used to
determine how much one unit will affect the others.
35. 35
3.3 THE DESIGNING AND CONSTRUCTION OF IIDSIPS
ANN are typically an imitation of neural structure of the brain , as its
crude electronic networks of neurons based on the neural structure of the brain.
The most important property of ANN is automatically learning/retrain the
co-efficient in the ANN according to the data input and outputs. Applying the
ANN approach to IDS we first have to expose the entire network to the normal
data, traffic, network parameters of desired environment and as well as types of
attacks and malwares , to allow the network to adjust accordingly and this
functionality will be provided during the learning phase, However there are a
few phases that are very essential to be carried out before the learning phase
-These Phases are:
Data monitoring
Pre-Processing
Feature Extraction
Classifier
The learning phase
Testing
Knowledgebase
Fig 3.5.Construction phases of ANN in IDS.
Data monitoring
Knowledgebase
Testing phasePre-processing Feature extraction classifier
The learning
phase
Learning
36. 36
DATA MONITORING: In this phase the a specific module will
monitor the data stream and capture the packets that will be use as a data source
for the IIDSIPS.
PRE-PROCESSING : This is the phase where the network traffic will be
collected and processed for use as input to the system.
FEATURE EXTRACTION: This module will extract feature vector from
the network packets(connection records) and will submit the feature vector to
the classifier module. The feature extraction process consist of feature
construction and feature selection. The quality of the feature construction and
feature selection algorithms is one of the most important factors that influence
the effectiveness of IIDSIPS
Classifier: Here we analyze the network stream and will draw a conclusion to
whether intrusion happens or not.
Learning phase: The learning phase is the process of optimization of the
parameters of the best set of connection coefficients(weights) for solving
problem are found.
As we discussed the types of IDSIPS before,there are two main types that
we will consider as prime objectives as implementation targets. The Network
Base and Host base , and also we will also consider the two main methods of
detection in both types of IDSIPS,which are Anomaly detection and Misuse
detection. The ANN will undergo different types of learning phases according to
the type of IDS desired in both host base and network base.
DIFFERENT TYPES OF LEARNING METHODS:
This section describes the learning ability of neural networks
One promising research in Intrusion Detection concerns the application of the
Neural Network techniques, for the misuse detection model and the anomaly
detection model.An artificial Neural Network consists of a collection of
37. 37
treatments to transform a set of inputs to a set of searched outputs, through a set
of simple processing units, or nodes and connections between them. Subsets of
the units are input nodes, output nodes, and nodes between input and output
form hidden layers; the connection between two units has some weight, used to
determine how much one unit will affect the other. Two types of architecture of
Neural Networks can be distinguished.
Supervised learning algorithms: where in the learning phase, the network
learns the desired output for a given input or pattern. The well known
architecture of supervised neural network is the Multi-Level Perceptron
(MLP);and mostly it is trained using the feedforward-backpropagation. The
MLP is employed for Pattern Recognition problems.
Unsupervised learning algorithms:
where in the learning phase, the network learns without specifying desired
output. Main example of this type of method is SOM( Kohonen self organized
map).
Both learning techniques are essentials to be carried out as each of the is
provided to be in use for different purposes,as we discussed before,that we will
use supervised learning to train the anomaly part of the IDS and will use the
unsupervised learning to train the mis use part of the IDS,as each holds specific
functionalities and deals with different problems and tasks,thus it takes both to
complete an efficient and a remarkably active IIDSIPS to be constructed.
SUPERVISED LEARNING ALGORITHM :There are several types of methods
to carry out this algorithmic learning that can be considered supervised,and the
main purpose of this learning is that the objectives and targeted objects and
events are specified thus comes the term supervised,as in being trained to do
specific tasks and obtain certain outcomes. The main methods of supervised
learning is Forwardpropagation and backwardpropagation and this type of
38. 38
training is for functionalities like pattern recognition and events monitoring,
Mostly used for ANOMALY DETECTION OF IDS
Forwardpropagation-Forwardpropagation is a supervised learning
algorithm and describes the "flow of information" through a neural net from its
input layer to its output layer.
The algorithm works as follows:
1. Set all weights to random values ranging from -1.0 to +1.0
2. Set an input pattern (binary values) to the neurons of the net's input layer
3. Activate each neuron of the following layer: - Multiply the weight values
of the connections leading to this neuron with the output values of the
preceding neurons - Add up these values - Pass the result to an activation
function, which computes the output value of this neuron
4. Repeat this until the output layer is reached
5. Compare the calculated output pattern to the desired target pattern and
compute an error value
6. Change all weights by adding the error value to the (old) weight values
7. Go to step 2
8. The algorithm ends, if all output patterns match their target patterns
Example:
Suppose you have the following 2-layered Perceptron:
Forwardpropagation in a 2-layered Perceptron
39. 39
Fig 3.6. Forwardpropagation in a 2-layered Perceptron
Patterns to be learned:
input target
0 1 0
1 1 1
First, the weight values are set to random values (0.35 and 0.81).
The learning rate of the net is set to 0.25.
Next, the values of the first input pattern (0 1) are set to the neurons of the input
layer (the output of the input layer is the same as its input).
The neurons in the following layer (only one neuron in the output layer) are
activated:
Input 1 of output neuron: 0 * 0.35 = 0
Input 2 of output neuron: 1 * 0.81 = 0.81
Add the inputs: 0 + 0.81 = 0.81 (= output)
Compute an error value by
subtracting output from target: 0 - 0.81 = -0.81
40. 40
Value for changing weight 1: 0.25 * 0 * (-0.81) = 0 (0.25 = learning rate)
Value for changing weight 2: 0.25 * 1 * (-0.81) = -0.2025
Change weight 1: 0.35 + 0 = 0.35 (not changed)
Change weight 2: 0.81 + (-0.2025) = 0.6075
Now that the weights are changed, the second input pattern (1 1) is set to the
input layer's neurons and the activation of the output neuron is performed again,
now with the new weight values:
Input 1 of output neuron: 1 * 0.35 = 0.35
Input 2 of output neuron: 1 * 0.6075 = 0.6075
Add the inputs: 0.35 + 0.6075 = 0.9575 (= output)
Compute an error value by
subtracting output from target: 1 - 0.9575 = 0.0425
Value for changing weight 1: 0.25 * 1 * 0.0425 = 0.010625
Value for changing weight 2: 0.25 * 1 * 0.0425 = 0.010625
Change weight 1: 0.35 + 0.010625 = 0.360625
Change weight 2: 0.6075 + 0.010625 = 0.618125
That was one learning step. Each input pattern had been propagated through the
net and the weight values were changed.
The error of the net can now be calculated by adding up the squared values of
the output errors of each pattern:
Compute the net error: (-0.81)2 + (0.0425)2 = 0.65790625
By performing this procedure repeatedly, this error value gets smaller and
smaller.
The algorithm is successfully finished, if the net error is zero (perfect) or
approximately zero.
41. 41
Backpropagation-Backpropagation is a supervised learning algorithm and is
mainly used by Multi-Layer-Perceptrons to change the weights connected to the
net's hidden neuron layer(s).
The backpropagation algorithm uses a computed output error to change
the weight values in backward direction. To get this net error, a
forwardpropagation phase must have been done before. While propagating in
forward direction, the neurons are being activated using the sigmoid activation
function.
The formula of sigmoid activation is:
1
f(x) = ---------, (1)
1 + e-input
The algorithm works as follows:
1. Perform the forwardpropagation phase for an input pattern and calculate
the output error
2. Change all weight values of each weight matrix using the formula
weight(old) + learning rate * output error * output(neurons i) *
output(neurons i+1) * ( 1 - output(neurons i+1) )
3. Go to step 1
4. The algorithm ends, if all output patterns match their target patterns
Example: Suppose you have the following 3-layered Multi-Layer-Perceptron:
42. 42
Fig3.7 .Back propagation in a 3-layered Multi-Layer-Perceptron
Patterns to be learned:
input target
0 1 0
1 1 1
First, the weight values are set to random values: 0.62, 0.42, 0.55, -0.17 for
weight matrix 1 and 0.35, 0.81 for weight matrix 2.
The learning rate of the net is set to 0.25.
Next, the values of the first input pattern (0 1) are set to the neurons of the input
layer (the output of the input layer is the same as its input).
43. 43
The neurons in the hidden layer are activated:
Input of hidden neuron 1: 0 * 0.62 + 1 * 0.55 = 0.55
Input of hidden neuron 2: 0 * 0.42 + 1 * (-0.17) = -0.17
Output of hidden neuron 1: 1 / ( 1 + exp(-0.55) ) = 0.634135591
Output of hidden neuron 2: 1 / ( 1 + exp(+0.17) ) = 0.457602059
The neurons in the output layer are activated:
Input of output neuron: 0.634135591 * 0.35 + 0.457602059 * 0.81 =
0.592605124
Output of output neuron: 1 / ( 1 + exp(-0.592605124) ) = 0.643962658
Compute an error value by
subtracting output from target: 0 - 0.643962658 = -0.643962658
Now that we got the output error, let's do the backpropagation.
We start with changing the weights in weight matrix 2:
Value for changing weight 1: 0.25 * (-0.643962658) * 0.634135591
* 0.643962658 * (1-0.643962658) = -0.023406638
Value for changing weight 2: 0.25 * (-0.643962658) * 0.457602059
* 0.643962658 * (1-0.643962658) = -0.016890593
44. 44
Change weight 1: 0.35 + (-0.023406638) = 0.326593362
Change weight 2: 0.81 + (-0.016890593) = 0.793109407
Now we will change the weights in weight matrix 1:
Value for changing weight 1: 0.25 * (-0.643962658) * 0
* 0.634135591 * (1-0.634135591) = 0
Value for changing weight 2: 0.25 * (-0.643962658) * 0
* 0.457602059 * (1-0.457602059) = 0
Value for changing weight 3: 0.25 * (-0.643962658) * 1
* 0.634135591 * (1-0.634135591) = -0.037351064
Value for changing weight 4: 0.25 * (-0.643962658) * 1
* 0.457602059 * (1-0.457602059) = -0.039958271
Change weight 1: 0.62 + 0 = 0.62 (not changed)
Change weight 2: 0.42 + 0 = 0.42 (not changed)
Change weight 3: 0.55 + (-0.037351064) = 0.512648936
Change weight 4: -0.17+ (-0.039958271) = -0.209958271
The same procedure is used for the next input pattern, but then with the changed
weight values.
45. 45
After the forward and backward propagation of the second pattern, one learning
step is complete and the net error can be calculated by adding up the squared
output errors of each pattern.
By performing this procedure repeatedly, this error value gets smaller and
smaller.
The algorithm is successfully finished, if the net error is zero (perfect) or
approximately zero.
Note that this algorithm is also applicable for Multi-Layer-Perceptrons with
more than one hidden layer. "What happens, if all values of an input pattern are
zero?"If all values of an input pattern are zero, the weights in weight matrix 1
would never be changed for this pattern and the net could not learn it. Due to
that fact, a "pseudo input" is created, called Bias that has a constant output value
of 1. This changes the structure of the net in the following way:
Fig3.8 .Backpropagation in a 3-layered Multi-Layer-Perceptron
46. 46
These additional weights, leading to the neurons of the hidden layer and the
output layer, have initial random values and are changed in the same way as the
other weights. By sending a constant output of 1 to following neurons, it is
guaranteed that the input values of those neurons are always differing from zero.
UNSUPERVISED LEARNING ALGORTIHM: This kind of learning
method is targeting the MIS USE DETECTION of the IDS ,it focuses on
approach of detecting attacks, it also define abnormal system behaviors at first
and then define the other behaviors as normal, It’s a learning technique without
any specified objectives nor targets,it acts more like a scout, Thus comes the
term unsupervised. And one of the main architectural structures of this method
is the SOM (kohonene self organizing map).
Self organization is an unsupervised learning algorithm used by the
Kohonen Feature Map neural net. As mentioned in previous sections, a neural
net tries to simulate the biological human brain, and selforganization is probably
the best way to realize this.
It is commonly known that the cortex of the human brain is subdivided in
different regions, each responsible for certain functions. The neural cells are
organizing themselves in groups, according to incoming informations.
Those incoming informations are not only received by a single neural cell, but
also influences other cells in its neighbourhood. This organization results in
some kind of a map, where neural cells with similar functions are arranged close
together. This selforganization process can also be performed by a neural
network. Those neural nets are mostly used for classification purposes, because
similar input values are represented in certain areas of the net's map.
47. 47
A sample structure of a Kohonen Feature Map that uses the selforganization
algorithm is shown below:
Fig3.9. Kohonen Feature Map with 2-dimensional input and 2-dimensional map
(3x3 neurons)
As you can see, each neuron of the input layer is connected to each
neuron on the map. The resulting weight matrix is used to propagate the net's
input values to the map neurons.
Additionally, all neurons on the map are connected among themselves. These
connections are used to influence neurons in a certain area of activation around
the neuron with the greatest activation, received from the input layer's output.
The amount of feedback between the map neurons is usually calculated using
the Gauss function:
-|xc-xi|2
feedbackci e = -------- , (2)
2 * sig2
48. 48
where
- xc is the position of the most activated neuron
- xi are the positions of the other map neurons
- sig is the activation area (radius)
In the beginning, the activation area is large and so is the feedback between the
map neurons. This results in an activation of neurons in a wide area around the
most activated neuron.
As the learning progresses, the activation area is constantly decreased and only
neurons closer to the activation center are influenced by the most activated
neuron.
Unlike the biological model, the map neurons don't change their positions on
the map. The "arranging" is simulated by changing the values in the weight
matrix (the same way as other neural nets do).
Because selforganization is an unsupervised learning algorithm, no input/target
patterns exist. The input values passed to the net's input layer are taken out of a
specified value range and represent the "data" that should be organized.
The algorithm works as follows:
1. Define the range of the input values
2. Set all weights to random values taken out of the input value range
3. Define the initial activation area
4. Take a random input value and pass it to the input layer neuron(s)
5. Determine the most activated neuron on the map: - Multiply the input
layer's output with the weight values - The map neuron with the greatest
resulting value is said to be "most activated" - Compute the feedback value
of each other map neuron using the Gauss function
6. Change the weight values using the formula: weight(old) + feedback value
* ( input value - weight(old) ) * learning rate
49. 49
7. Decrease the activation area
8. Go to step 4
9. The algorithm ends, if the activation area is smaller than a specified value
Programming the ANN using JAVA: APPENDIX A
The main Class “NeuralNet”
This class is the first class in the structure.
abstract class NeuralNet Extends::java.lang.Object
boolean displayNow ()
Indicates, whether the net should be drawn or not, depending on its display step. True, if
the net should be drawn. False otherwise.
boolean finishedLearning ()
Indicates that the net has finished learning. True, if the learning process is finished. False
otherwise.
String getElapsedTime ()
Returns the elapsed learning time of a neural net.
int getLearningCycle ()
Returns the current learning cycle of a neural net.
double getLearningRate ()
Returns the current learning rate of a neural net.
int getMaxLearningCycles ()
Returns the number of maximum learning cycles of a neural net.
void resetTime ()
50. 50
Resets the net's learning time.
void setDisplayStep ( int displayStep )
Sets a value that indicates the interval to display the net.
void setLearningRate ( double learningRate )
Sets the learning rate of a neural net.
void setMaxLearningCycles ( int maxLearningCycles )
Sets the number of learning cycles, the net shall perform. If -1, the net has no maximum
cycle.
Class BackpropagationNet
This class represents a Backpropagation Net neural net.
public class BackpropagationNet
Extends:: java.lang.Object NeuralNet
Instantiated by:: Application
Constructors::public BackpropagationNet ()
Methods
void addNeuronLayer ( int size )
Adds a neuron layer with size neurons.
Note that neuron layers are sequentially added to the net.
51. 51
void connectLayers ()
Connects all neuron layers with weight matrices.
Must be called after all neuron layers have been added.
double getAccuracy ()
Returns the accuracy value.
double getError ()
Returns the current error of the net.
String getInputPattern ( int patternNr )
Returns the input pattern with number patternNr.
double getMinimumError ()
Returns the minimum error of a neural net.
float[] getNeuronOutputs ( int layerNr )
Returns the output values of all neurons in layer layerNr.
int getNumberOfLayers ()
Returns the number of neuron layers.
int getNumberOfNeurons ( int layerNr )
Returns the number of neurons in layer layerNr.
int getNumberOfPatterns ()
Returns the number of patterns.
int getNumberOfWeights ()
Returns the number of weights of all weight matrices.
52. 52
int getNumberOfWeights ( int matrixNr )
Returns the number of weights in weight matrix matrixNr.
String getOutputPattern ( int patternNr )
Returns the output pattern with number patternNr.
float getPatternError ( int patternNr )
Returns the error of output pattern patternNr.
String getTargetPattern ( int patternNr )
Returns the target pattern with number patternNr.
float[][] getWeightValues ( int matrixNr )
Returns the weight values of weight matrix matrixNr.
The values for matrixNr start with zero!
void learn ()
Performs one learning step.
synchronized void readConversionFile ( String conversionFileName )
Reads a conversion table for ASCII-binary values from file conversionFileName.
synchronized void readPatternFile ( String patternFileName )
Reads input and target patterns from file patternFileName.
String recall ( String recallInput )
Tries to recall the correctoutput for a learned input pattern recallInput.
void setAccuracy ( double accuracy )
Sets an accuracy value for the net, which is something like a "fuzzy border" for
output/recall purposes (default is 0.2).
53. 53
void setMinimumError ( float minimumError )
Sets the minimum error of a neural net.
Class KohonenFeatureMap
This class represents a Kohonen Feature Map neural net.
public class KohonenFeatureMap
Extends:: java.lang.Object NeuralNet
Instantiated by Application
Constructors ::public KohonenFeatureMap ()
Methods:
void connectLayers ( InputMatrix inputMatrix )
Connects the feature map and the input layer (which is generated depending on the size
of the inputMatrix) with a weight matrix.
void createMapLayer ( int xSize, int ySize )
Creates a two-dimensional feature map with xSize*ySize map neurons.
double getActivationArea ()
Returns the current activation area.
double getInitActivationArea ()
Returns the initial activation area.
double getInitLearningRate ()
Returns the initial learning rate.
int getMapSizeX ()
Returns the number of neurons in the map layer's x-dimension.
54. 54
int getMapSizeY ()
Returns the number of neurons in the map layer's y-dimension.
int getNumberOfWeights ()
Returns the number of weights in the weight matrix.
double getStopArea ()
Returns the final activation area.
float[][] getWeightValues ()
Returns the weight values of the net's weight matrix.
void learn ()
Performs a learning step.
void setInitActivationArea ( double initActivationArea )
Sets the initial activation area.
void setStopArea( double stopArea )
Sets the final activation area at which the net stops learning.
void setInitLearningRate ( double initLearningRate )
Sets the initial learning rate.
55. 55
3.4 HOW DOES IIDSIPS WORK (METHOD DESCRIPTION)
After the system is being trained to function for all its purposes,and his includes both
sides of the system, the anomaly detection and the mis use detection ,now we will
consider a set up for the hard ware in a network topology with full scale protection using
IDS sensors and other components.
Fig .3.10 implementation of IDS in large scale network
SERVER
SWITC
H
IDS
MANAGER Ids collector
Idssensor
Idssensor
FIRE
WALL switch
router
internet
TAP TAP
Networkhistorydatabase
56. 56
ANOMALY DETECTION TECHNIQUES
Anomaly detection [4] is based on a host or network. Many distinct
techniques are used based on type of processing related to behavioral model.
They are: Statistical based, Operational or threshold metric model, Markov
Process or Marker Model, Statistical Moments or mean and standard deviation
model, Univariate Model, Multivariate Model, Time series Model, Cognition
based, Finite State Machine Model, Description script Model, Adept System
Model, Machine Learning based, Bayesian Model, Genetic Algorithm model,
Neural Network Model, Fuzzy Logic Model, Outlier Detection Model,
Computer Immunology based, User Intention based. Here in this paper, only the
few Machine Learning Techniques are discussed.
Packet Monitor
This module monitors network stream real time and capture packets to
serve for the data source of the
NIDS. The packet capture library provides a high level interface to packet
capture system. All packets
on the network, even those destined for other hosts are accessible through this
mechanism.
Pre-processor
In preprocessing phase, network traffic collected and processed for use as
input to the system.
Feature Extraction
This module extracts feature vector from the
network packets (connection records) and submits the feature vector to the
classifier module. Feature
extraction is an important part of a pattern
57. 57
recognition system. The feature extraction process consists of feature
construction and feature selection. The quality of the feature construction and
feature
selection algorithms is one of the most important factors that influence the
effectiveness of IDS.
Achieving reduction of the number of relevant traffic
features without negative impact on classification accuracy is a goal that largely
improves the overall
effectiveness of the IDS. Most of the feature construction as well as feature
selection works in
intrusion detection practice is still carried out through manually utilizing domain
knowledge.
Classifier
The function of this module is to analyze the network stream and to draw
a conclusion whether
intrusion happens or not. Neural network classifiers perform very successfully
for recognizing and
matching complicated or incomplete patterns. The most successful application
of neural network is
classification or categorization and pattern recognition. The learning process is
essentially an
optimization process in which the parameters of the best set of connection
coefficients (weighs) for
solving a problem are found and includes the following basic steps [9]:
Present the neural network with a number of inputs.
58. 58
Check how closely the actual output generated for a specific input matches
thedesired output.
Change the neural network parameters to better approximate the outputs.
Decision When detecting that intrusion happens, this module will send a
warning message to the user.
Knowledgebase
This module serves for the training samples of the classifier phase. The
Artificial Neural Networks can work effectively only when it has been
trained correctly and sufficiently. The intrusion samples can be perfected under
user participation, so the capability of the detection can improve continually.
All of these modules together make the NIDS architecture system based on the
artificial neural networks. The present study is aimed to solve a multi class
problem in which not only the attack records are distinguished from normal
ones, but also the attack type is identified.
Fig 4.1. the IIDS system architecture
nNETWORK
CLASSIFIER
FEATURE
PREPROCESSOR
PACKET
DECISION
TRAINING
KNOWLEDGE BASE
59. 59
4 RESULTS OF USING INTELLIGENT INTRUSION DETECTIUON AND
PREVENTION SYSTEM
4.1 ADVANTAGES AND DISADVATANGES
Advantages of Neural Network
The first advantage in the utilization of a neural network in the detection
of instances of misuse would be the flexibility that the network would provide.
A neural network would be capable ofanalyzing the data from the network, even
if the data is incomplete or distorted. Similarly, thenetwork would possess the
ability to conduct an analysis with data in a non-linear fashion. Both of these
characteristics is important in a networked environment where the information
which is received is subject to the random failings of the system. Further,
because some attacks may be conducted against the network in a coordinated
assault by multiple attackers, the ability to process data from a number of
sources in a non-linear fashion is especially important.
The inherent speed of neural networks is another benefit of this approach.
Because the protection of computing resources requires the timely identification
of attacks, the processing speed of the neural network could enable intrusion
responses to be conducted before irreparable damage occurs to the system.
Because the output of a neural network is expressed in the form of a probability
the neural lnetwork provides a predictive capability to the detection of instances
of misuse. A neural network-based misuse detection system would identify the
probability that a particular event, or series of events, was indicative of an attack
against the system. As the neural network gains
60. 60
experience it will improve its ability to determine where these events are likely
to occur in the attack process. This information could then be used to generate a
series of events that should occur if this is in fact an intrusion attempt. By
tracking the subsequent occurrence of these events the system would be capable
of improving the analysis of the events and possibly conducting defensive
measures before the attack is successful.
However, the most important advantage of neural networks in misuse
detection is the ability of the neural network to "learn" the characteristics of
misuse attacks and identify instances that are unlike any which have been
observed before by the network. A neural network might be trained to recognize
known suspicious events with a high degree of accuracy. While this would be a
very valuable ability, since attackers often emulate the "successes" of others, the
network would also gain the ability to apply this knowledge to identify instances
of attacks which did not match the exact characteristics of previous intrusions.
The probability of an attack against the system may be estimated and a potential
threat flagged whenever the probability exceeds a specified threshold.
Disadvantages of Neural Network
There appear to be two primary reasons why neural networks have not
been applied to the problem of misuse detection in the past. The first reason
relates to the training requirements of the neural network. Because the ability of
the artificial neural network to identify indications of an intrusion is completely
dependent on the accurate training of the system, the training data and the
training methods that are used are critical. The training routine requires a very
large amount of data to ensure that the results are statistically accurate. The
training of a neural network for misuse detection purposes may require
thousands of individual attacks sequences, and this quantity of sensitive
information is difficult to obtain.
61. 61
4.2 CONCLUSION
Research and development of intrusion detection systems has been
ongoing since the early 80's and the challenges faced by designers increase as
the targeted systems because more diverse and complex. Misuse detection is a
particularly difficult problem because of the extensive number of vulnerabilities
in computer systems and the creativity of the attackers. Neural networks provide
a number of advantages in the detection of these attacks.
Many methods have been employed for intrusion detection. However,
modeling networking traffic for a simple representation to a neural network
shows great promise, especially on an individual attack basis. Also, using SOMs
as a clustering method for MLP neural networks is an efficient way of creating
uniform, grouped input for detection when a dynamic number of inputs are
present. Once trained, the neural network can make decisions quickly,
facilitating real-time detection. Neural Networks using both supervised and
unsupervised learning have many advantages in analyzing network traffic and
the apporach will be a continuing area of research.
The new reality in cyber security is that network breaches are inevitable,
and the ability to monitor and control access and behavior patterns and misuse
relies upon intrusion detection and prevention methods to be more quickly
identified and more effectively addressed. An IDS/IPS is a must-have device; an
ANN model based on the learning patterns and techniques and classifying
intrusion data packets is an effective approach. The main advantages of the
ANNs over traditional IDSs are their abilities to learn, classify, process
information faster, as well as their ability of self-organization. For these reasons,
62. 62
Neural Networks can increase the accuracy and efficiency of IDSs and AI
techniques can improve IDS/IPS effectiveness.
63. 63
BIBLOGROPHY
[1]Anderson, D., Frivold, T. & Valdes, A (May, 1995). Next-generation Intrusion
Detection Expert System (NIDES):
[2] Cramer, M., et. al (1995). New Methods of Intrusion Detection using Control-Loop
Measurement. In Proceedings of the Technology in Information Security Conference
(I'ISC) '95
[3] Debar, H., Becke, M.,& Siboni, D. (1992). A Neural
Network Component for an Intrusion Detection System. In Proceedings of the IEEE
Computer Society Symposium on Research in Security and Privacy.
[4] Debar, H. & Dorizzi, B. (1992). An Application Recurrent Network to an Intrusion
DetectionSystem. In Proceedings of the International Joint Conference on Neural
Networks. pp. (11)478-483.
[5] Denning, Dorothy. (February, 1987). An Intrusion-Detection
Model. IEEE Transactions on Software Engineering, Vol. SE-13, NO.2
.
[6] Fox, Kevin L., Henning, RhondaR., and Reed, Jonathan H. (1990). A Neural Network
. Approach Towards Intrusion Detection. In Proceedings of the 13th National
ComputerSecurity Conference.
[7] Frank, Jeremy. (1994). Artificial Intelligence and Intrusion Detection: Current and
Future
Directions. In Proceedings of the 17th National Computer Security Conference.
[8] Helman, P. and Liepins, G., (1993). Statistical foundations of audit trail analysis for
the detection of computer misuse, IEEE Trans. on Software Engineering, 19(9):886-901
. [9] Kumar, S. & Spafford, E. (1994) A Pattern Matching Model for Misuse Intrusion
Detection. In Proceedings of the 17th National Computer Security Conference, pages 11-
21.
[10] Kumar,S.&Spafford, E. Software Architecture to SupportMisuse Intrusion
Detection.Department of Computer Sciences, Purdue University; CSD-TR-95-009
64. 64
[11] Lunt, T.F. (1989). Real-Time Intrusion Detection. Computer Security Journal Vol.
VI, Number 1pp 9-14.
[12] Ryan, J., Lin, M., and Miikkulainen, R. (1997). Intrusion Detection with Neural
Networks. AI Approaches to Fraud Detection and Risk Management: MAl Workshop
(Providence, RhodeIsland), pp. 72-79.
[13] Sebring, M., Shell house, E., Hanna, M. & Whitehurst, R. (1988) Expert Systems in
Intrusion Detection:
[14] Stanford-Chen, S. (1995, May 7). Using Thumbprints toTrace Intruders. UC Davis.
[15] Tan, K. (1995). The Application of Neural Networks to UNIX Computer Security.
In Proceedings of the IEEE International Conference on Neural Networks, Vol.]
[16]Brecht, D. (2010, April 15). Network Intrusion Detection Systems: a 101. Retrieved
from http://www.brighthub.com/computing/smb-security/articles/38389.aspx#imgn_1
[17]Compare Business Products (2014, March 18). Security: IDS vs. IPS Explained.
Retrieved from http://www.comparebusinessproducts.com/fyi/ids-vs-ips
[18]GCN. (2014, December 9). What’s next in cybersecurity automation. Retrieved from
http://gcn.com/articles/2014/12/09/dhs-ease.aspx
[19]Infosecurity Magazine. (2011, October21). Small enterprises are suffering more
intrusions, survey finds. Retrieved from http://www.infosecurity-
magazine.com/news/small-enterprises-are-suffering-more-intrusions/
[20]InfoSight Inc. (n.d). Intrusion Detection (IDS) & Intrusion Prevention (IPS).
Retrieved from http://www.infosightinc.com/IT-Security/IDS_IPS.php
[21]Kashyap, S. (2013, May). Importance of Intrusion Detection System with its
Different approaches. Retrieved from
http://www.ijareeie.com/upload/may/24_Importance.pdf
[22]Kumar, A. (2014, May). Intrusion detection system using Expert system (AI) and
Retrieved from http://www.ijarcsms.com/docs/paper/volume2/issue5/V2I5-0064.pdf
65. 65
[23]Mukhopadhyay, I. (2014). Hardware Realization of Artificial Neural Network Based
Intrusion Detection & Prevention System. Retrieved from http://file.scirp.org/Html/3-
7800230_50045.htm