Network dialog minimization and network dialog diffing: Two novel primitives for network security applications

1. Network Dialog Minimization and Network Dialog Diffing: Two Novel Primitives of Network Security
M. ZubairRafique
zubair.rafique@cs.kuleuven.be
Juan Caballero (IMDEA Software Institute)
Christophe Huygens (iMinds-Distrinet, KU Leuven)
WouterJoosen(iMinds-Distrinet, KU Leuven)

2. Network Trace
Malicious SIP INIVTE Request
VoIP Phones
PCs
SIP Servers
Network Switch
Gateway Router
Internet
Server Crashed

3. Attack traffic?

4. Drive-by Download Milkers
Downloads a malware sample
Browser plugin detected and vulnerabilities exploited
Redirects to exploit kit landing page
Navigate to given URL
HoneyClient
•Grier et al. “Manufacturing Compromise: The Emergence of Exploit-as-a-Service”, CCS 2012
•Nappaet al. “Driving in the Cloud: An Analysis of Drive-by Download Operations and Abuse Reporting”, DIMVA 2013
Downloads a malware sample
Minimized Dialog, IPs, Time
Milker

5. PCAP
PCAP
PCAP
PCAP
PCAP
Unlabeled Malware Samples
Malware Network Dialogs
Compare Dialogs
PCAP
PCAP
PCAP
PCAP
PCAP
Cluster 1
Cluster 2
Cluster 3
•Perdisciet al. “Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces”, Computer Networks
•Rafiqueet al. “Firma: Malware clustering and network signature generation with mixed network behaviors”, RAID 2013
Dialog Clustering

6. In a nutshell …
●Problem
-Network Dialog Minimization
-Network Dialog Diffing
●Applications
-Building drive-by download milkers
-Cookie expiration validation
-Simplifying user interfaces
-Vulnerability analysis
-Dialog clustering
●Outcomes
-Reduction in time and bandwidth
-Perfect precision and high recall

7. Outline
●Network Dialog Minimization
●Network Dialog Diffing
●Evaluation and Findings
-Milkersfor 9 exploit kits (14000 malware samples)
-17% top websites allow cookie replay >1 month
-Savings of time per year and employee
-New vulnerability in SIP server
-Clustering 6 malware families (F-Meausre= 87.6%)
●Limitations and Future Improvements

8. Network Dialog Minimization:“Given an original dialog that satisfies a goal, can we produce a minimized dialog comprising the smallest subset of the original dialog that when replayed still achieves the same goal as the original dialog?”
Network Dialog Minimization

9. ●Encode network dialog as dialog tree.
Dialog Generation
C2
C1
C3
M1
M2
M3
M4

10. Exploit
kit
Pre-filtering
Filtered
Nodes
C:M:F
C:M:F
IPs
Blackhole 1.x
73
6:6:60
5:5:50
2
CoolExploit
646
18:58:569
5:5:49
2
CritiXPack
192
4:19:168
2:7:62
2
Eleonore
936
12:76:848
8:66:736
2
Phoenix
132
12:12:107
7:7:73
1
ProPack
137
10:12:114
6:6:57
2
RedKit
154
8:17:128
2:6:57
1
Serenity
54
5:5:43
5:5:43
1
Unknown
79
5:7:66
5:7:66
2
Dialog Generation
Building Drive-by Download Milkers

11. Architecture

12. Network Delta Debugging
Test Dialog
Replay
Remove Dialog
Yes
No
Original Dialog
Minimized Dialog
Keep Dialog
Goal

13. C2
C1
C3
M1
M2
M3
M4
C2
C3
M2
M4
Network Delta Debugging

14. Network Delta Debugging
●Generalized version of delta debugging
-Reset Button
-Goal beyond crashing the program
-Hierarchical structure of dialog tree
Zeller et al. “Simplifying and isolating failure-inducing input”, IEEE Transactions in Software Engineering.
•NDM deals with remote networked applications.
-commercial Virtual Network (VPN) that offers exit points in more than 50 countries (4500 IPs)
Incorrect Minimization

15. L1
L2
L3
Tree
IPs
GDT
Time
C:M:F
C:M:F
C:M:F
Nodes
used
Pref.
(sec.)
2:2:22
2:2:22*
2:2:6
11
33
157.0
1:1:7
1:1:7*
1:1:3
6
15
X
42.5
1:4:33
1:1:7
1:1:3
6
17
X
49.0
1:1:8
1:1:8*
1:1:4
7
27
X
215.8
1:1:7
1:1:7*
1:1:3
6
15
X
24.2
1:1:7
1:1:7*
1:1:3
6
15
X
37.3
2:6:57
2:2:19
2:2:10
15
71
250.4
2:2:15
2:2:15*
2:2:6
11
28
X
79.7
1:2:14
1:1:7
1:1:3
6
18
X
51.0
Exploit
kit
Blackhole 1.x
CoolExploit
CritiXPack
Eleonore
Phoenix
ProPack
RedKit
Serenity
Unknown
Network Delta Debugging
Building Drive-by Download Milkers

16. Network Dialog Diffing

17. Network Dialog Diffing:“Given two dialogs, identifying how similar they are, how to align them, and how to identify their common and different parts?”
Network Dialog Diffing
Rock.in
Rock.in
Dialog 1
Dialog 2
4 RRP
3 RRP

18. sim(D1, D2) = (1/N) * Σ wi
sim(D1, D2) = (0.9+1+1+0)/4
= 2.9/4
= 0.725
i=1
N
Dialog Similarity

19. Evaluation and Findings

20. 34 times faster than honey client.
14000 malware downloaded from single machine.
Drive-by Download Milkers
Results Summary
Cookie Expiration Validation
71 times reduction in replay time. Savings of 20 hours of processing/day.
31% of websites allows cookie replay (on logout). 17% cookies live over a month.
Simplifying User Interface
Savings of 3 hours per employee per year.
Command line tool to perform building task.
Vulnerability Analysis
Finding new vulnerability in OpenSBCServer OSVDB 86607 (See details in the paper).
Dialog Clustering
Benign Dialogs (F-Measure = 100%), Malware Dialogs (F-Measure = 87.6%)

21. Results Summary
34 times faster than honey client.
14000 malware downloaded from single machine.
Drive-by Download Milkers
Cookie Expiration Validation
71 times reduction in replay time. Savings of 20 hours of processing/day.
31% of websites allows cookie replay (on logout). 17% cookies live over a month.
Simplifying User Interface
Savings of 3 hours per employee per year.
Command line tool to perform building task.
Vulnerability Analysis
Finding new vulnerability in OpenSBCServer OSVDB 86607 (See details in the paper).
Dialog Clustering
Benign Dialogs (F-Measure = 100%), Malware Dialogs (F-Measure = 87.6%)
OSVDB: 86607

22. 34 times faster than honey client.
14000 malware downloaded from single machine.
Drive-by Download Milkers
Results Summary
Cookie Expiration Validation
71 times reduction in replay time. Savings of 20 hours of processing/day.
31% of websites allows cookie replay (on logout). 17% cookies live over a month.
Simplifying User Interface
Savings of 3 hours per employee per year.
Command line tool to perform building task.
Vulnerability Analysis
Finding new vulnerability in OpenSBCServer OSVDB 86607 (See details in the paper).
Dialog Clustering
Benign Dialogs (F-Measure = 100%), Malware Dialogs (F-Measure = 87.6%)
Clustering Results
Dataset
Algor.
Clusters
Precision
Recall
F-Measure
Alexa
PAM
30
100%
100%
100%
Malware
PAM
10
100%
64.8%
78.6%
Alexa
Agg.
30
100%
100%
100%
Malware
Agg.
12
100%
78.0%
87.6%

23. Limitations and Future Improvements
●Minimized dialog may look suspicious
●Dynamically generated requests
●Achieving global minimum
●Diffing of dialogs beyond HTTP

24. Conclusion
●Introduce the problem of network dialog minimizationand present novelnetwork delta debuggingtechnique.
●Propose a noveldialog diffing technique.
●Applied our techniques to 5 different applications.
-building drive-by download milkers
-cookie expiration validation
-simplifying user interfaces
-vulnerability analysis
-dialog clustering

25. Questions?