INSECS: Intelligent networks security system

INSECS - INTELLIGENT NETWORK
SECURITY SYSTEM
Thesis by: Nadun N Rajasinghe
Supervisors: Dr. Jagath Samarabandu and Dr. XianbinWang
Department of Electrical and Computer Engineering,
The University ofWestern Ontario,
London.

Security issues in Modern distributed networks
■ Insecure web interface - Using default password and having insecure password recovery method
■ Insufficient or non-existent authentication/authorization
■ Vulnerable network services – communication ports have known vulnerabilities they will be exploited by hackers
■ Lack of transport encryption
■ Insecure software/firmware - software needs to be verified by signatures
■ Poor physical security
■ Data Breaches and Loss
■ Account or Service Hijacking
■ Malicious Insiders
■ HypervisorVulnerabilities – This will cause intruder to gain access to all virtual machines
■ Denial of Service

Different defensive
strategies
Firewall
Encryption
Anti-Virus
Authentication
Intrusion
Detection/Prevention

What is an Intrusion Detection System
■ An Intrusion Detection System is a device or software that monitors a system for malicious
activity.

How does an Intrusion Detection System work?
■ There are two popular detection strategies.
■ Anomaly Detection ( Behavior based) – Model normal behavior.Anything deviating
from that is an intrusion.
■ Misuse Detection ( Signature based) – Identify details about attacks and create attack
signatures. If traffic match these, it is considered an intrusion.
■ IDS can be further classified based on the Scope of operation.
■ Network-based (NIDS) – Intrusion Detection which relies on network traffic
information.
■ The usual place to have these is inside the network after the firewall.
■ Sometime these are also placed in selected places inside the network.
■ Host-based (HIDS) - Intrusion Detection based on system activity in a host.
■ These are placed at each host.

How does an Intrusion Detection System work?
■ There are two popular detection strategies.
■ Anomaly Detection – Model normal behavior.Anything deviating from that is an
intrusion.
■ Misuse Detection – Identify details about attacks and create attack signatures. If
traffic match these, it is considered an intrusion.
■ IDS can be further classified based on the Scope of operation.
■ Network-based (NIDS) – Intrusion Detection which relies on network traffic
information.
■ The usual place to have these is inside the network with the firewall.
■ Sometime these are also placed in selected places inside the network.
■ Host-based (HIDS) - Intrusion Detection based on system activity in a host.
■ These are placed at each host.

Areas of Research Interest
■ Looked at 2 aspects of intrusion Detection
■ Problems with Intrusion Datasets
■ Intrusion Detection System for Modern Distributed networks including IoT and Cloud

EXISTING CHALLENGES
AND
CONTRIBUTIONS OFTHIS
RESEARCH

Challenges
in Intrusion
Datasets
■ IDS rely heavily on Datasets for training or learning. If the Dataset is faulty,
so is the IDS.
■ A standard dataset is created for conditions available at that time
■ TheVersions of software and services
■ The nature of attacks and attack tools
■ Dataset has attributes, the dataset creators thought were good
■ Users of the dataset has no choice on customizability
■ Once dataset is created and released it’s fixed
■ No Choice on the Output format of Dataset
o PCAP in most cases, sometimes CSV
■ No choice over the attributes
o Ex. NSL-KDD has 41 attributes that cannot be changed

Dataset
Creation
■ INSecS-DCS - an on-demand intrusion dataset creation software that can run
on a network of choice and gives the user the ability to make fully customized
datasets.
1. A conference Paper – INSecS-DCS:A highly customizable Intrusion dataset
creation framework (CCECE, Quebec City 2018)
2. Available for Public use under an MIT license for the advancement of Intrusion
Detection research.
Does not
work!

Challenges
of Intrusion
Detection in
Modern
Networks
■ TheWidely Used Intrusion Detection Systems do not support Distributed
Intrusion Detection.
■ HighTraffic overhead leads to Lower Intrusion Detection Speeds
■ Detecting Multi stage attacks
Traffic
Volume
Detection
Speed

Challenges
of Intrusion
Detection in
Modern
Networks
■ Intrusion Detection systems need Adaptive learning to save detection effort
■ Limited Computing power in IoT devices.
■ Many IoT devices have low computing power
■ InternalAttacks
■ not all IDS architectures are able to detect internal attack.
IDS
Network
Data
Rule updates

Intrusion
Detection
■ INSecS-IDS – a flexible Intrusion Detection System fit for modern networks
due to it’s design and architecture, which is aimed at supporting IoT, Cloud and
Distributed systems.
■ A journal Paper – ACMAn IDS with Hierarchical Decision Making within a
Framework for Intelligent Network Security System
■ A survey of Intrusion detection as applied to IoT, Cloud and other distributed
networks.
■ A Flexible IDS design that allows third party Applications.
■ A comparison of the performance of INSecS-IDS compared to Snort.
■ Possible example setups for INSecS-IDS in IoT and Cloud applications.

Intelligent Network
Security Systems
Intrusion Detection
Framework
(INSecS-IDF)

INSecS-DCS
Dataset
Creation
Software

INSecS-IDS
Intrusion
Detection
System

 The Learning module extracts the rules out
of the dataset to be used in the IDS.
Learning
Module
 In this research the learning modules
functionality is simulated human expertise

RELATEDWORK AND
BACKGROUND
INSECS-
DCS

Dataset Creator
■ ID2T:A DIY dataset creation toolkit for Intrusion Detection Systems [56]
■ They have addressed some of the problems like
 Ability to create Datasets locally, anytime
 Insert attacks of your choice
 Attribute selection
■ Still not addressed,
 Get processed Dataset (not just RAW PCAP)
 Make new attributes withTime properties
 Change attributes

Processed Dataset
Packet
Extract
Packet
information
Packet
Extract
Overall
Information
Packet
Packet
Raw Dataset

Processed Dataset
■ Even in the datasets that provide these kind of attributes, you cannot make new ones or
customize them.
Destination
IP
Source IP
Last 100
 There are some datasets like Kyoto and NSL-KDD that offer these kind of processed attributes
but you cannot
1. change things like the number “100”.
2. Cannot add anyTemporal property to it

IMPLEMENTATION
OF INSECS-DCS
INSECS-
DCS

DCS
Architecture
TShark is a network protocol analyzer
• It lets you capture packet data from a live network
• Read packets from a previously saved capture file
• TShark's native capture file format is pcap format, which is also the
format used by tcpdump and various other packet analyzer
Packet
Capturing
Packet Pre-
Processing
Collecting individual
packet information
Collecting overall packet
information for time
window
Dividing traffic into
time windows
Raw Dataset Processed Dataset

Packet Pre-
processing
Packet
Capturing
Packet Pre-
Processing
packet information
window
time windows
• Convert the Captured packets into a format suitable for the algorithm to
use easily. In this case, a dictionary of key value pairs.

Collecting
Individual
packet
information
Packet
Capturing
Packet Pre-
Processing
packet information
window
time windows
• Information from individual packets are collected by selecting key value pairs of
interest.These include,
o Protocols used -TCP, UDP, IP, FTP, SMTP, SSH, SSL,ARP, DHCP,HTTP
o Source and destination information - IP address, port numbers

Dividing
traffic flow
into time
windows
+
collecting
overall
features
Packet
Capturing
Packet Pre-
Processing
packet information
window
time windows
• Select a time window and analyzing the traffic flow during that time. (
customizable time window)
• As opposed to getting information from just the individual packets here we get
information on overall traffic behavior during the time window and identify
common trends in traffic

Dividing
traffic flow
into time
windows
+
collecting
overall
features  Before INSecS-DCS, you had to do this manually to the entire dataset if you
wanted more attributes.
 This allows users to create attributes like the following listed below
Attribute Description
connection pairs The number of different source and destination pairs
num ports number of different port numbers used
src bytes the total amount of source traffic
tcp frame length the total amount of frame bytes forTCP traffic
udp length the total amount of UDP data
num ssl total number of packets containing SSL traffic

PCAP file CSV or txt
Packet
Capturing
Packet Pre-
Processing
packet information
window
time windows
Processed
and Raw
Datasets

INSecS-DCS
■ To show the capability of the dataset creator, I used the PCAP files from the ISCX dataset and made
a processed dataset out of it.
■ The ISCX dataset is updated once a year.Therefore, it is a somewhat updated dataset.
■ This experiment was only to show the capability of INSecS-DCS. I still maintain the argument that
the best dataset for you is the dataset you create for your network.
■ INSecS-DCS is provided for public use under an MIT license
■ Hosted on GitHub - https://github.com/nrajasin/Network-intrusion-dataset-creator

Comparison
of DCSWith
ID2T toolkit
Capability INSecS-DCS ID2T
Ability to Label dataset Yes Yes
Open Source Yes Yes
Raw PCAP dataset Yes Yes
Has a GUI No Yes
Allows attack injection within the software No Yes
Ability to divide traffic into time window and get overall
traffic attributes
Yes No
Ability to select input method ( packets captured on a
network of choice or get a raw PCAP dataset from another
source )
Yes No
Processed dataset that can fed intoWEKA and other ML
tools directly
Yes No
Attribute selection for processed dataset Yes No

 The Dataset creator is capable of making
custom datasets that fit the network.
 The IDS is capable of providing Intrusion
Detection in a distributed manner, handling
large volumes of data.
 Signature based IDS on detection
strategy, that works with a Rule set.
 It is a Network based IDS on detection
scope but has nodes at each host in
network.
 The Learning module extracts the rules out
of the dataset.

RELATEDWORK AND
BACKGROUND
INSECS-IDS

■ Snort:A free, widely used Intrusion Detection/ Intrusion Prevention System
(IDS/IPS). It is a rule-based IDS/IPS which has a large rule base.
■ Bro:An open-source, passive, Unix based network intrusion detection
system that works well with large traffic loads.A key advantage Bro has is
its deep analysis involving a broad range of protocols.
■ Suricta: Considered to be a close competitor to Snort, with an advantage
over Snort in detection speed. According to literature, Suricta supports
Snort rules which makes it a strong performer as Snort is the most widely
supported IDS.
■ OSSEC:An Open Source, host-based Intrusion Detection System that
works for most Operating Systems. It performs log analysis, file integrity
checking,Windows registry monitoring, etc, within the host and
communicates using encrypted channels with the OSSEC server for
intrusion detection.
Widely Used Intrusion Detection Systems
Does not directly support
Distributed setup
Does not directly support
Distributed setup
Speed comes at the cost
of resources and also
does not support
distributed setups
Host Based architecture
does not offer network
wide protection

Existing solutions
■ Nezarat [36], has proposed using game theory to model the scenario involving a detection
agent, reporting from different nodes in the distributed system, and an intruder.
■ An IDS capable of detecting DoS attacks for distributed client-server systems using separate
client detectors and server detectors has been proposed by Kshirsagar et al. [13].The method
uses a rule-based approach to detecting attacks but details of rule generation are not explained.
■ By running different instances of Snort in a distributed manner, at different nodes across the
network, researchers have controlled the load on each of them by careful resources usage
observation [64]. However, due to processing all the instances with multiple Snort instances, the
researchers were not able to complete the experiment because of resource insufficiency.
■ Noorman et al. [38] has proposed an embedded security system with a separate microprocessor
that relies on knowing each node in the IoT system.The isolation capabilities ensure that even if
one node is compromised, intruders will not be able to hack into the other nodes using the
compromised node.
■ Lee et al. [24] proposed a distributed IDS solution based on analyzing the energy consumption at
each node, to detect anomalies.

INSecS – Intrusion Detection System
These are the main features of the proposed IDS
■ Designed for Distributed operation
■ Less detection delay with Large data volumes
■ Less resource usage
■ Adaptive rule updating
■ Ability to detect multi-step attacks faster
■ Detect internal attacks

IMPLEMENTATION
OF INSECS-IDS
INSECS-IDS

Logical Architecture – Hierarchical Decision Making
Primary
Decision Node
Complex
Decision
Engine
Primary
Decision Node
Primary
Decision Node
Only traffic that is suspected of being part of
attack is sent for more processing
This is different from the traditional approach
because there all traffic is checked for all
attacks.

IDS controller
The IDS Controller is responsible for
initiating the Packet handler and the initial
Primary Decision Nodes.
The initial PDNs can be detectors for
reconnaissance attacks or the most
common attacks.
More can be activated based on the
network behavior.

Packet Handler
As soon as the packet handler is initiated,
it runs Tshark, which captures the stream
of packets going in and out of the node
and generates an object with all the
packet attributes and values for each and
every packet.
This object is then sent to a pre-
processing unit in order to generate a
Python dictionary, which is broadcasted to
all the active buffers.
There is a buffer for each active PDN and
the corresponding PDN reads the stream
of dictionaries continuously.

Data Buffers
These Units function as Queues for the
streams of packets into the Primary
decision node engine. These prevent the
different decision units interfering with
each other’s operations.

Primary Decision
Node -1
This unit is responsible for determining
whether a packet is a possible threat or
not. For example, in the Portscan PDN, all
packets with SYN flag set to ON, are
marked as possible attack packets.
This reduces the traffic load on the
Intrusion detection process because
packets which have no association with
the attack are not subjected to further rule
checks.
There is a PDN at each network node.
Since there are perform lightweight tasks,
they can even be placed in low resource
devices like IoT.

Primary Decision
Node - 2
The traffic suspected of belonging to an
attack, is then sent to the Complex
Decision Engine using HTTP.
This choice was made because it is a very
common protocol and can be used easily
for the purpose.

Complex Decision
Engine
The complex decision engine performs the
complex rule and condition checks to make
final decision on whether there is an intrusion
or not.
This unit is placed in a device with a lot of
system resources as it handles all the data
streams from all the PDNs in the network.
The complex processes include,
• Dividing traffic streams into time
windows
• Counting instances of Different
incidents
• Condition and rule checks
• Determining if there is an intrusion

Multi Stage attack
If a reconnaissance attack is detected, the IDS
assumes that another attack would follow.
The CDE would then initiate new PDNs to
detect possible attacks if they are not already
switched on.

IDS adaptive behavior
Once an attack is detected, the IDS gets
updated with all the details of the attack.
This includes information like,
• The Source IP
• The ports used
• Geo location if available
If an attack were to originate from the same
source, it can be detected immediately
because of this updating process.

Experiment- 1
■ An experimental setup was used to validate the proposed setup.
■ 3 PDNs were used. Portscan, Ipsweep, Smurf.
■ The CDE was setup on another Server in the same network.
■ The DARPA 98 dataset was used as the traffic source.
■ The test cases were as follows:
Test case 1:Test the validity of using hierarchical decision making for faster Intrusion
detection
Two setups with identical features were used.The only difference was, one setup had hierarchical
decision making. I selected 3 dataset files containing the smurf attack. Only one detector was activated.
Test case 2: Ensure the results of test 1 are repeatable for multiple attacks
Two similar setups but with 3 PDNs and one dataset file containing 3 attacks (Smurf, Portscan, IPsweep)
were used.

Experiment - 2
■ Another feature of INSecS-IDS is the flexible design.The functionality of the CDE can be achieved
using a Complex Event Processor.
■ A Complex Event Processor is a software that can process streams of realtime data.
■ The advantage of this is, users who are used to SQL like queries can easily use the IDS.
■ The disadvantage would be features like adaptive rule updating would not work unless the CEP is an
adaptive one.
■ The CEP picked for this experiment was an adaptive CEP designed and implemented by a fellow lab
member.
■ The same 2 test cases mentioned before were used to validate this claim.

Experiment - 3
■ To test the effectiveness of INSecS-IDS compared to an industry grade IDS, another experiment
was devised.
■ Snort, (a popular IDS described earlier) was used to detect the same 3 attacks from the same
dataset files as Experiment 1.

Detection Results for INSecS-IDS
Week 1,Wednesday 55.6 5.090
Week3,Wednesday 37.9 6.837
Week 5, Monday 150.65 124.266
Experiment 1 –Test Case 1
Smurf 124.621 14.202
IP sweep 124.616 14.202
Port scan 124.613 14.202
■ The same accuracy was observed in all test cases. Meaning that all the attack
instances listed by the dataset creators were detected by the IDS.

Week 1,Wednesday 55.2 5.474
Week3,Wednesday 37.9 7.240
Week 5, Monday 150.65 133.58
Experiment 2–Test Case 2
Smurf 124.621 14.9
IP sweep 124.616 14.9
Port scan 124.613 14.9
■ The same accuracy was observed in all test cases. Meaning that all the attack
instances listed by the dataset creators were detected by the IDS.
Detection Results for INSecS-IDS

Comparing Snort with INSecS-IDS
■ Experiment 3 showed that Snort also detected the same attack instances identified by
INSecS.
■ There was no method to compare the detection time because snort did packet reading and
Intrusion Detection on the same computer.There was no way to get the level of distributed
functionality that INSecS has.
■ Snort has too many rules, with multiple rule bases. Sometimes there are overlapping rules for
the same function. In comparison INSecS-IDS rule base is well structured to avoid duplication.

Conclusions for DCS and IDS
■ INSecS-DCS can be used for making customizable datasets for any network.
■ INSecS-IDS performs accurate Intrusion Detection in a distributed environment with low system
resources.
■ Hierarchical decision making provides a clear advantage in detection time and large traffic
volume handling.

Example IoT setup
IoT
Primary
Decision
Node
Mobile device
Primary
Decision
Node
Server
Primary
Decision
Node
IDS
controller
Complex
Decision
Engine
The CDE and IDS controller can be
in device with considerable
computing resources.
The IoT devices can even be in
different networks.
Each IoT device can have PDNs
activated.

Example Cloud setup
Server
Primary
Decision
Node
IDS
controller
Complex
Decision
Engine
Server
Primary
Decision
Node
Server
Primary
Decision
Node
The servers can be hosted in
different networks. Still the
proposed IDS can be deployed.
Each Server has PDNs activated in
them. Any suspicious traffic is
forwarded to the CDE.
CDE can be hosted on a device
with considerable computing
capabilities.

Improvements for DCS and IDS
■ The communication between the distributed nodes is done using HTTP. I plan to use
encrypted communication to prevent the IDS itself being compromised.
■ The DatasetCreator and the IDS needs to be connected to provide a better IDS
solution. Right now, a human expert has to go through the Datasets and make the
rules. By implementing the learning module, I plan to make that an automated
process.
■ The learning module will generate rules for the specific network, based on the
Datasets created by the INSecS-DatasetCreation Software.

INSECS: Intelligent networks security system

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to INSECS: Intelligent networks security system

Similar to INSECS: Intelligent networks security system (20)

Recently uploaded

Recently uploaded (20)

INSECS: Intelligent networks security system

Editor's Notes