Shusei tomonaga pac_sec_20171026

Neural Network for
Detecting APT
Lateral Movement
Shusei Tomonaga
JPCERT/CC
PacSec 2017

Copyright ©2017 JPCERT/CC All rights reserved.
Self-introduction
 Analysis Center at JPCERT/CC
 Malware analysis, Forensics investigation.
 Written up posts on malware analysis and technical
findings on this blog and Github.
̶ http://blog.jpcert.or.jp/
̶ https://github.com/JPCERTCC/aa-tools
※ Iʼm a malware analyst, not a data scientist.
1
Shusei Tomonaga

Difficult to prevent a network from being
compromised
The focus of attention is shifting towards early
detection of lateral movement
Development of system that records and
analyzes the behavior of processes on the host
is flourishing now
EDR(Endpoint Detection and Response) is
receiving attention
2
Trend of APT Incident Detection

Copyright ©2017 JPCERT/CC All rights reserved.3
Goals of This Presentation
Proposal of a method of
detecting lateral movement

Approach
 The details of the lateral movement method are
unknown
 Investigate the method of lateral movement and
create a detection method
4
In order to detect lateral movement, it is necessary
to know how an attacker spreads infection.

This Presentation Topics
5
１
Research of
Lateral Movement
２ Pattern of Lateral Movement
３
Detecting Lateral Movement
using Machine Learning
４ Detection System

１
Research of
Lateral Movement
３

Research of Lateral Movement
Investigating C&C servers and malware connections
in five operations.
 APT10 (named by FireEye)
 Dragon OK (named by Palo Alto)
 Blue Termite (named by Kaspersky)
 Tick (named by Symantec)
7
Research Methods

Research of Lateral Movement
 Investigate how to spread infection from the
commands executed by the attacker.
8
Research Methods

APT incident investigated by JPCERT/CC
BKDR_ChChes (APT10)
Asruex
Elirks
Tick
Blue Termite
Scanbox
Winnti
APT17
2013 2014 2015
1 4 7 10 1 4 7 10 1 4 7 10
2016
1 4 7 10
9

Data Set
Total command
execution: 16,866
Total number of
infected host: 645
10
Research Overview

Data Set
Total command
execution: 16,866
Total number of
infected host: 645
11
Research Overview
Total Windows command execution: 14,268

Tools Used by Attackers at Lateral Movement
 Why attackers use Windows commands and
legitimate tools?
 They are not detected by antivirus software.
12
Attackers use not only attack tools
but also Windows commands and legitimate tools.

１
Research of
Lateral Movement
３

AD/
File Server
Target Network
1. Infection
2. Initial
investigation 3. Internal reconnaissance
4. Spread of infection
5. Sending stolen data
Overview of APT Incident and Lateral Movement
6. Delete evidence

Lateral Movement: Initial Investigation
 The most used command is tasklist.
 If the infected host was a virtual machine for
analysis, the attacker will escape soon.
15
•  Collect information of the infected host
Initial investigation

Windows Command Used for Initial Investigation
Rank Command Count
1 tasklist 327
2 ver 182
3 ipconfig 145
4 net time 133
5 systeminfo 75
6 netstat 42
7 whoami 37
8 nbtstat 36
9 net start 35
10 set 29
11 qprocess 27
12 nslookup 11
16

Lateral Movement: Internal Reconnaissance
•  Look for information saved in the compromised
machine and information on the network
Internal Reconnaissance
17
  The most used command is dir.
— The attacker look around confidential data
stored in the infected host.
  For searching the local network, net is used.

Windows Command Used for Internal Reconnaissance
Rank Command Count
1 dir 4466
2 ping 2372
3 net view 590
4 type 543
5 net use 541
6 echo 496
7 net user 442
8 net group 172
9 net localgroup 85
10 dsquery 81
11 net config 32
12 csvde 21
18

net Command
  net view
— Obtain a list of connectable domain resources
  net user
— Manage local/domain accounts
  net localgroup
— Obtain a list of users belonging to local groups
  net group
— Obtain a list of users belonging to certain domain groups
  net use
— Access to resources
19

Lateral Movement: Spread of Infection
•  Infect the machine with other malware or
try to access other hosts
Spread of infection
20
  The most used command is at.
— “at” command is not supported on Windows 10,
Windows 8.1 etc.
— If “at” don’t exist, schtasks is used.
  Password dump tool is always used.

Windows Command Used for Spread of Infection
21
Rank Command Count
1 at 445
2 move 399
3 schtasks 379
4 copy 299
5 ren 151
6 reg 119
7 wmic 40
8 powershell 29
9 md 16
10 runas 7
11 sc 6
12 netsh 6

Remote Command Execute Used Windows Command
22
at command
> at [IP Address] 12:00 cmd /c "C:windowstemp
mal.exe"
schtasks command
> schtasks /create /tn [Task Name] /tr C:1.bat /sc
onstart /ru System /s [IP Address]

Remote Command Execute Used Windows Command
23
wmic command
> wmic /node:[IP Address] /user:”[User Name]” /
password:”[PASSWORD]” process call create
“cmd /c c:WindowsSystem32net.exe user”

Lateral Movement: Delete Evidence
•  Delete files used by the attacker and logs
Delete evidence
24
  The most used command is del.
  For deleting the event log, wevtutil is used.

Windows Command Used for Delete Evidence
Rank Command Count
1 del 844
2 taskkill 80
3 klist 73
4 wevtutil 23
5 rd 15
25

•  Research the attack pattern of lateral movement
•  Attacker uses Windows command for lateral
movement
•  Lateral movement can be detected by
monitoring Windows command execution
26
Summary so Far

Blacklist
Scoring
Machine learning
27
Method of Monitoring Malicious Windows Command Execution

Blacklist
Detect execution of commands that are likely used
by an attacker
Command executed
by the attacker
at
whoami
del
net use
Command
at
schtasks
klist
net use
…
Black List
whoami
tasklist
del
dir
Command executed
by the attacker
False NegativeDETECTED!

 Detect execution of net use, schtasks and at command
 These commands may be executed by an application or
user and cannot be blacklisted
29
Blacklist
Can't detect an attack unless blacklisted commands
are executed
issue

Scoring
Scoring executed Windows commands and
detected when it is executed above the threshold
Command Score
at 50
schtasks 50
whoami 10
net use 30
del 5
tasklist 10
Score sheet
at
whoami
del
net use
Calculation result
50 + 30 + 10 + 5 = 95
Threshold = 90
DETECTED!
Command executed
by the attacker

Scoring
 Scoring Windows commands according to importance
 The cost of creating a scoring table is high (Constant update
required)
 If an attacker executes a command with a low score, it can not
be alerted
issue
Scoring executed Windows commands and alerted
when it is executed above the threshold


32
Machine Learning
Detect maicious Windows command execution
using machine learning
Machine Learning
at
whoami
del
net use
Result
DETECTED!
Command executed
by the attacker

Machine Learning
Detect anomaly Windows command
using machine learning
After this slide, I present the observation
results of this method

１
Research of
Lateral Movement
３

 Supervised learning
 Unsupervised learning
 Reinforcement Learning
35
Machine Learning
Machine learning is a field of computer science that
gives computers the ability to learn without being
explicitly programmed. - Wikipedia※ -
※ https://en.wikipedia.org/wiki/Machine_learning
I use this

Collection of training data
Data cleansing
Data analysis with Machine Learning
Evaluate and select the best
algorithm
36
Flow of Algorithm Selection for Machine Learning

Using data from five attack campaigns analyzed by
research of lateral movement
 Dragon OK (named by Palo Alto)
 Blue Termite (named by Kaspersky)
 Tick (named by Symantec)
37
Collection of Training Data

Data cleansing
algorithm
38

Data Cleansing
> cd intellogs
> whoami
> klist
> net use
> klist purge
> ping -n 1 10.1.44.16
> ping -n 1 10.1.2.16
> net use 10.1.2.16
> dir 10.1.2.16c$users
> copy bb.bat 10.1.2.16c$windowssystem32
> net time 10.1.2.16
> at 10.1.2.16 12:27 bb.bat
> dir 10.1.2.16c$windowssystem32inf.txt
> move 10.1.2.16c$windowssystem32inf.txt .
> del 10.1.2.16c$windowssystem32bb.bat
What to learn?

Data Cleansing
> cd intellogs
> whoami
> klist
> net use
> klist purge
> ping -n 1 10.1.44.16
> ping -n 1 10.1.2.16
> net use 10.1.2.16
> dir 10.1.2.16c$users
> copy bb.bat 10.1.2.16c$windowssystem32
> net time 10.1.2.16
> at 10.1.2.16 12:27 bb.bat
> dir 10.1.2.16c$windowssystem32inf.txt
> move 10.1.2.16c$windowssystem32inf.txt .
> del 10.1.2.16c$windowssystem32bb.bat
Commands executed
on the host (No argument)
Use the executed command
set as one data

Data cleansing
at whoamidel net use
tasklist dir netsh
whoami schtasks echo
dir del echo whoami
Command Set 1
Command Set 2
Command Set 3
Command Set 4

Creating Training Data
Learning Commands
tasklist ver ipconfig net time cd systeminfo
netstat whoami nbtstat net start set qprocess
nslookup fsutil net view type net use echo
net user net group net localgroup dsquery net config csvde
net share quser net session query user tracert nltest
at move schtasks copy ren reg
wmic powershell md cscript runas sc
netsh wusa icacls del taskkill klist
wevtutil rd
42
Commands to be learned are narrowed down to 50,
which are often executed by attackers
in lateral movement based on our research

Creating Training Data

Data cleansing
algorithm
44

Decision
tree
Random
forest
Neural
network
Bayesian
network
k-means SVM
45
Data Analysis
Example of algorithm

Data Analysis
Algorithm to evaluate
Decision
tree
Random
forest
Neural
network
Bayesian
network
k-means SVM

T F
0.2 0.8
47
Bayesian Network
Probabilistic model that expresses the causal
relation between "cause" and "result" by graph
structure and probability
Rain
WetGrass
Sprinkler
Rain T F
F 0.4 0.6
T 0.01 0.99
Sup Rain T F
F F 0 1
F T 0.8 0.2
T F 0.9 0.1
T T 0.99 0.01

Bayesian Network
 When the value of the variable is decided, the probability
distribution of the unknown variable can be derived
 Predict the future
Probabilistic model that expresses the causal
relation between "cause" and "result" by graph
structure and probability

 When a command is executed, it predicts whether it is an
attack or not
 Creating network model by learning
49
Detect Anomaly Windows Command Using Bayesian Network
at
ATTACK
net use

Model created

Created a model
> net use 10.1.2.16
> copy bb.bat 10.1.2.16c$windows
system32
> at 10.1.2.16 12:27 bb.bat
Flow of malware execution
The flow of Lateral Movement is properly modeled by
learning

Neural Network
Network model that mimics the structure of the human brain
Often used for image recognition
3
Input Output 1 2 3
4 5 6
7 8 9Handwritten digits
Neural Network

Detect Anomaly Windows Command Using Neural Network
Input: executed command
Output: attack or not
Input Output
Neural Network
at
whoami
del
net use
True
or
False
Attack?

Design of Neural Network
 Feedforward neural network
— Input: 50 commands
— Output: Attack or not（True or False）
54
Affine
Batch
Norm
ReLU Affine
Soft
max
Command
Set
3-layers
True
False

Learning Result by Neural Network
0.4
0.5
0.6
0.7
0.8
0.9
1
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
accuracy
epoch

Problem
56
・ The model created by learning is a black box,
and it is unknown which criteria is used
・ The criteria changes for each learning result
Problem of neural network

Evaluation Criterion of Neural Network
 Visualizing network
 Variable importance
 Activation Maximization
 Sensitivity Analysis
 Local Interpretable Model-Agnostic Explanations
57
There is research to investigate criteria of
neural network
I use this

Variable Importance for Neural Networks
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
dsquery
cscript
netsh
tracert
nltest
nslookup
wusa
nbtstat
ren
fsutil
set
echo
netstat
copy
wevtutil
icacls
net_share
move
net_session
md
cd
schtasks
csvde
net_start
net_time
type
ver
systeminfo
whoami
ipconfig
tasklist
klist
qprocess
runas
net_use
reg
sc
net_config
rd
net_user
taskkill
wmic
quser
del
at
powershell
net_localgroup
net_view
net_group
query
attack
command

Data cleansing
algorithm
59

Evaluation Indices
Recall Those predicted as "true" among "true" results
Precision "True" results among those judged as "true"
F-
measure
Evaluation index of prediction accuracy
60
Recall= 𝑇 𝑃/𝑇𝑃
+ 𝐹𝑁 
Precision= 𝑇 𝑃/𝑇𝑃
+ 𝐹𝑃 
F-measure=2 𝑅𝑒𝑎𝑙𝑙∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛/𝑅𝑒𝑐𝑎𝑙𝑙
+ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛

Result
Algorithm recall precision F-measure
Bayesian Network 0.994343 0.683800 0.810337
Neural network 0.965517 0.967742 0.966628
Decision tree 0.839154 0.960347 0.895669
Random forest 0.915388 0.975964 0.944705
61
Average value repeated 1,000 times

Bayesian Network
•  Low false negative rate
Neural network
•  The balanced algorithms
Random forest
•  Low false positive rate
62
Result

１
Research of
Lateral Movement
３

System Overview

System Overview (Client)
65
•  Collect commands executed via cmd.exe
cmdlogs.bat (Shell Script)
•  Send the collected logs to the server
•  Confirm analysis result and display alert
Invoke-DetectLM.ps1 (PowerShell)

System Overview (Server)
66
•  Save the logs
Elasticsearch
•  Visualize the log
Kibana
•  Collect logs saved in Elasticsearch, detect malicious command
execution by machine learning
DetectLM.py (Python)

DetectLM.py (Python)
 Analyze the logs using neural network
 Data exchange with Elasticsearch via REST API
67
Collect logs saved in Elasticsearch, detect
malicious command execution by machine learning

Alert Level
Logs sent from the hosts
Anomaly?
Alert Level 0
No
Alert Level 1
Yes
User reported as
malicious?
Alert Level 2
NoYes

•  Default
Level 0
•  Logs detected as malicious by machine learning
Level 1
•  Error log reported by user
Level 2
69
Alert Level
The log has three levels of detection level

Notice to Clients
 User then can set a “ignore flag” when it’s false positive.
70
Users will be notified of logs
detected as malicious (Alert Level: 1)

Kibana Dashboard
71

Demo
72

Future Work
Update algorithm
•  Current algorithm does not consider time series
data and frequency of execution
•  Some commands have to be executed in order
•  Take in (consider) time series of execution
73

Conclusion
Windows commands are used during lateral
movement
Lateral movement can be detected by monitoring
malicious Windows command execution
Machine learning assist detecting malicious
command execution
74

Thank you
Q&A
https://github.com/JPCERTCC/DetectLM

Shusei tomonaga pac_sec_20171026

More Related Content

What's hot

Viewers also liked

Similar to Shusei tomonaga pac_sec_20171026

More from PacSecJP

Recently uploaded

Shusei tomonaga pac_sec_20171026